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Abstract 


HUMAN. A/S 


This  report  reviews  research  literature  pertaining  to  models  of  human  information  aggregation  in 
order  to  apply  them,  when  possible,  to  combat  identification  (CID)  decisions  made  by  individuals 
in  a  land  force  context.  In  particular,  the  review  examined  social  science,  science  and  military 
literature  to  identify  principles  and  theories  associated  with  how  human  decision  makers  weigh  and 
integrate  information.  The  review  identified  a  variety  of  heuristics  and  cognitive  biases  associated 
with  these  processes,  along  with  formal  and  informal  models  of  combining  qualitative  and 
quantitative  information.  The  report  presents  current  conceptualizations  of  combat  identification 
and  places  it  within  the  context  of  cue-based  decision  making  under  the  intuitive  decision 
framework.  By  framing  this  complex  decision  we  were  able  to  evaluate  the  applicability  of  the 
methods  of  aggregation  to  the  CID  task. 
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Resume 


Le  present  rapport  examine  la  documentation  de  recherche  portant  sur  les  modeles  de 
regroupement  de  1’  information  humain  afm  de  les  appliquer,  lorsque  cela  est  possible,  aux 
decisions  en  matiere  d’ identification  au  combat  (IDCbt)  prises  par  des  personnes  dans  un  contexte 
des  forces  terrestres.  Plus  particulierement,  on  a  examine  les  publications  des  sciences  sociales, 
scientifiques  et  militaires  pour  cemer  les  principes  et  les  theories  lies  a  la  maniere  dont  les 
decideurs  evaluent  et  integrent  l’information.  L’examen  a  permis  de  relever  des  connaissances 
heuristiques  et  des  preconceptions  cognitives  bees  a  ces  processus,  ainsi  que  des  modeles  officiels 
et  informels  pour  combiner  de  l’information  qualitative  et  quantitative.  Le  rapport  presente  les 
conceptualisations  actuelles  de  1’  identification  au  combat  et  situe  cette  demiere  dans  le  contexte  de 
la  prise  de  decisions  fondees  sur  des  indices  selon  le  cadre  de  decisions  intuitives.  En  exprimant 
cette  decision  complexe,  nous  avons  pu  evaluer  l’applicabilite  des  methodes  de  regroupement  a  la 
tache  d’IDCbt. 
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This  report  reviews  the  results  of  a  keyword  search  of  the  research  literature  associated  with 
models,  theories  and  principles  of  human  information  aggregation  and  cue-based  decision  making 
as  they  relate  to  combat  identifications  made  by  individuals  in  a  land  force  context.  The  goals  of 
this  review  were  to: 

•  Perform  searches  of  scientific  and  military  literature  databases; 

•  Obtain  relevant  literature  pertaining  to  formal  models  of  information  aggregation,  human 
information  aggregation  research,  and  psychological  models  of  human  cue-based  decision¬ 
making; 

•  Draft  a  report  documenting  models  of  information  aggregation,  summarizing  major 
principles  and  theories  of  human  information  aggregation  and  developing  a  categorization 
scheme  in  which  to  evaluate  the  applicability  of  models  of  information  aggregation  to 
human  decision  making. 

The  search  resulted  in  approximately  65  primary  articles  from  science,  social  science  (Economics, 
Political  Science,  Organizational  Behaviour,  and  Psychology)  and  military  databases.  These 
articles  were  reviewed  to  acquire  major  themes  and  principles  associated  with  the  weighing  and 
combining  of  information. 

The  report  contains  a  review  of  the  following  themes  and  concepts: 

•  Conceptualization  of  combat  identification 

•  Decision  making  frameworks 

•  Models  of  information  aggregation 

o  Reasons  for  aggregating  information 
o  Methods  of  information  integration 
o  Cue  weighing  strategies 
o  Information  presentation  effects 

By  categorizing  the  methods  of  information  aggregation  based  on  their  associated  decision  making 
frameworks,  we  were  able  to  determine  which  models  may  be  most  applicable  to  combat 
identification.  In  particular,  we  identified  several  heuristics  and  cognitive  biases  as  being 
specifically  relevant  to  the  tasks  of  weighing  and  integrating  information.  This  review  highlights 
these  methods  to  improve  understanding  of  the  aggregation  process. 
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Sommaire 


Le  present  rapport  examine  les  resultats  d’une  recherche  par  mots  cles  effectuee  dans  les  etudes 
liees  aux  modeles,  aux  theories  et  aux  principes  du  regroupement  de  1’ information  humain  et  de  la 
prise  de  decisions  fondee  sur  les  indices  en  ce  qui  a  trait  a  1’ identification  au  combat  effectuee  par 
des  personnes  dans  un  contexte  des  forces  terrestres.  Les  objectifs  du  present  examen  sont  les 
suivants  : 

•  Effectuer  des  recherches  dans  les  bases  de  donnees  de  publications  scientifiques  et 
militaires; 

•  Obtenir  des  publications  pertinentes  relativement  aux  modeles  officiels  de  regroupement  de 
1’  information,  a  la  recherche  sur  le  regroupement  de  E  information  humain  et  aux  modeles 
psychologiques  de  prise  de  decisions  fondee  sur  les  indices; 

•  Rediger  un  rapport  qui  decrit  les  modeles  de  regroupement  de  l’information,  dans  lequel  on 
resume  les  principes  et  theories  essentiels  du  regroupement  de  l’information  humain  et 
presente  un  schema  de  classification  qui  permet  d’evaluer  l’applicabilite  des  modeles  de 
regroupement  de  l’information  a  la  prise  de  decisions  chez  l’humain. 

La  recherche  a  releve  environ  65  articles  principaux  des  bases  de  donnees  des  sciences,  des 
sciences  sociales  (economic,  science  politique,  comportement  organisationnel  et  psychologie)  et 
militaires.  Ces  articles  ont  ete  examines  pour  cerner  les  principaux  themes  et  principes  lies  a 
E analyse  et  au  regroupement  de  l’information. 

Le  rapport  comprend  un  examen  des  themes  et  concepts  suivants  : 

•  Conceptualisation  de  l’identification  au  combat 

•  Cadres  de  prise  de  decisions 

•  Modeles  du  regroupement  de  l’information 

o  Raisons  pour  regrouper  1’ information 

o  Methodes  pour  regrouper  1’ information 

o  Strategies  d’analyse  des  indices 

o  Effets  de  la  presentation  de  1’ information 

En  classant  les  methodes  de  regroupement  de  l’information  selon  leurs  cadres  de  prises  de 
decisions  connexes,  nous  avons  ete  en  mesure  de  determiner  les  modeles  qui  s’appliquent  le  mieux 
a  l’identification  au  combat.  Nous  avons  notamment  releve  plusieurs  connaissances  heuristiques  et 
preconceptions  cognitives  particulierement  adaptees  aux  taches  d’analyse  et  de  regroupement  de 
1’ information.  Cet  examen  met  en  valeur  ces  methodes  afm  d’ameliorer  la  comprehension  du 
processus  de  regroupement. 
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1.1  Scope  and  Objectives 

“..the  human  mind  still  retains  the  role  of  merging ,  pattern  recognition  and  deciding...  As 
ID  techniques  become  more  numerous  and  sophisticated,  the  burden  on  human  merging 
and  deciding  is  becoming  excessive...  ”  (U.S.  Defense  Science  Board,  1996) 

This  report  documents  the  results  of  a  survey  of  scientific  literature  pertaining  to  theories  and 
models  of  human  information  aggregation.  The  purpose  of  this  review  is  to  better  understand  how 
decision-makers  weigh  and  integrate  information  from  a  variety  of  sources  in  the  course  of  making 
a  decision.  The  review  focuses  on  the  aggregation  process  as  opposed  to  the  information-gathering 
phase  or  the  final  step  of  choosing  a  course  of  action.  However,  the  principles  and  theories 
associated  with  the  models  are  reviewed  in  the  context  of  cue-based  decision-making,  therefore 
requiring  a  summary  of  the  different  decision  making  frameworks.  This  literature  review  is 
restricted  to  human  information  aggregation  models  because  of  the  goal  to  evaluate  how 
individuals  (e.g.,  commander,  dismounted  soldier)  might  combine  information  from  multiple 
sources  when  making  a  combat  identification  (CID)  in  a  land  force  context.  Computational 
models,  often  referred  to  as  data  fusion  algorithms,  therefore,  are  beyond  the  scope  of  this  review. 
Literature  from  psychology,  social  sciences,  and  military  databases  are  examined  to  identify  a 
comprehensive  set  of  models  of  information  aggregation. 

The  specific  objectives  of  the  report  are: 

•  To  perform  searches  of  scientific  and  military  literature  databases; 

•  Obtain  relevant  literature  pertaining  to  formal  models  of  information  aggregation, 
human  information  aggregation  research,  and  psychological  models  of  human  cue- 
based  decision-making; 

•  Draft  a  report  documenting  models  of  information  aggregation,  summarizing  major 
principles  and  theories  of  human  information  aggregation  and  developing  a 
categorization  scheme  in  which  to  evaluate  the  applicability  of  models  of  information 
aggregation  to  human  decision  making. 


1.2  Outline  of  Report 

This  report  begins  with  a  detailed  description  of  the  search  methodology  used  to  acquire  the  articles 
for  review.  Then,  the  literature  review  commences  in  Section  3  with  a  conceptualisation  of  combat 
identification,  followed  by  background  information  related  to  the  different  decision  making 
frameworks  (Section  4).  Section  5  details  the  purposes  of  integrating  multiple  pieces  of 
information,  provides  a  review  of  the  different  models  of  human  information  aggregation,  and 
discusses  weighting  strategies  and  issues  related  to  the  methods  of  information  presentation.  The 
review  concludes  in  Section  6  with  a  categorization  of  the  most  likely  methods  of  information 
aggregation  utilized  during  the  combat  identification  decision. 

1.3  Contract 

This  work  was  performed  under  contract  W771 1-067996/001/TOR.  The  Scientific  Authority  (SA) 
for  this  work  was  Dr.  David  Bryant. 
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2.1  Keywords 

We  developed  a  set  of  keywords  for  the  literature  search  based  on  our  experiences  with  the  two 
areas  of  interest  for  this  review:  1)  models,  principles  and  theories  related  to  information 
aggregation,  and  2)  psychological  models  of  human  cue-based  decision-making.  In  addition, 
search  terms  for  combat  identification  were  formed  to  locate  literature  to  assist  in  framing  this 
review. 

The  keywords  were  dividing  into  several  categories  (see  Table  1),  including  a  general  category 
used  to  narrow  the  search  when  searches  returned  a  large  number  of  records. 


Table  1.  Categories  and  Associated  Keywords 


Category 

Keyword 

Related  Keywords 

Models  of  information  aggregation 

Information  aggregat* 

Information  integration,  information 
fusion,  decision  making,  decision 
bias,  heuristic,  uncertainty, 
probability,  weighting,  evidence 
integration,  cue  integration,  multiple- 
cue  judgment 

Cue-based  decision-making 

Recognition  prim* 

Recognition  primed  decision  making, 
RPD,  RPDM,  cue-based,  mental 
simulation,  situation  assessment, 
situation  awareness,  pattern 
matching 

Naturalistic  decision  making 

NDM,  intuitive  decision  making, 
intuition,  uncertainty,  dynamic 
decision  making,  time  pressure, 
satisficing,  Lens  model 

General 

Combat  identification,  combat  ID, 

CID,  situation  assessment,  threat 
assessment,  defence,  defense, 
military,  army,  decision  making, 
information,  knowledge,  judgment 

Searches  revealed  several  key  authors  ( 
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Table  2)  involved  in  research  relevant  to  the  categories  of  interest.  For  example,  researchers  in  the 
area  of  information  aggregation  were  identified  through  the  proceedings  from  Wallsten’s  (2004) 
“workshop  on  information  aggregation  in  decision  making”.  When  necessary  to  find  more  specific 
information,  these  names  were  used  to  enhance  the  search. 
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Table  2.  Key  Researchers  in  the  Areas  Defined  by  the  Literature  Review 


Models  of  Information  Aggregation: 

David  V.  Budescu,  University  of  Illinois 

Ronald  R.  Yager,  Machine  Intelligence  Institute,  Iona  College,  New  York 
llan  Yaniv,  Hebrew  University,  Israel 

Decision  Making  Heuristics  and  Biases: 

Amos  Tversky,  Stanford  University  and  Daniel  Kahneman,  Princeton  University 
Gerd  Gigerenzerand  Daniel  G.  Goldstein,  Max  Planck  Institute  for  Human  Development 
Cue-based  decision-Making: 

Gary  Klein,  Klein  Associates  Inc.,  Ohio 
Raanan  Lipshitz,  Haifa  University,  Israel 
Eduardo  Salas,  University  of  Central  Florida 


2.2  Databases 

Searches  for  relevant  literature  were  conducted  of  the  following  databases  and  sources: 

•  PsycINFO 

•  PsycARTICLES 

•  Scientific  and  Technical  Information  Network  (STINET) 

•  Canada  Institute  for  Scientific  and  Technical  Information  (CISTI) 

•  National  Technical  Information  Services  (NTIS) 

•  Institute  of  Electrical  and  Electronics  Engineers  (IEEE)  Xplore 

•  Human  Factors  and  Ergonomics  Society  (HFES) 

•  PubMed 

•  Defence  Research  &  Development  Canada  (DRDC)  Defence  Research  Reports 

•  World  Wide  Web  (WWW),  Google  Scholar 

PsycINFO  is  a  department  of  the  American  Psychological  Association  (APA)  that  offers  products 
to  aid  researchers  in  locating  psychological  literature.  Paid  membership  with  the  APA  is  required 
for  use  of  this  database  online  at  www.apa.org/psycinfo.  The  database  is  based  on  Psychological 
Abstracts  and  contains  non-evaluative  summaries  of  literature  in  psychology  and  related  fields 
(e.g.,  human  factors,  education,  business  and  social  studies).  The  database  contains  over  one 
million  electronically  stored  bibliographic  references  including  journal  articles,  dissertations, 
reports  and  book  chapters. 

PsycARTICLES  is  a  database  of  full-text  articles  from  journals  published  by  the  APA,  APA 
Educational  Publishing  Foundation,  Canadian  Psychological  Association  and  Hogrefe  Publishing 
Group.  An  APA  membership  is  required  for  use  of  this  database  located  online  at 
www.apa.org/psycarticles/. 
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STINET  is  a  publicly-available  database  (stinet.dtic.mil)  which  provides  access  to  citations  of 
documents  such  as:  unclassified  unlimited  documents  that  have  been  entered  into  DTIC's  Technical 
Reports  Collection  (e.g.,  dissertations  from  the  Naval  Postgraduate  School),  the  Air  University 
Library  Index  to  Military  Periodicals,  Staff  College  Automated  Military  Periodical  Index,  DoD 
Index  to  Specifications  and  Standards,  and  Research  and  Development  Descriptive  Summaries. 

The  full-text  electronic  versions  of  many  of  these  articles  are  also  available  from  this  database. 

CISTI  is  the  library  for  the  National  Research  Council  of  Canada  and  a  world  source  for 
information  in  science,  technology,  engineering  and  medicine.  The  database  is  searchable  on-line  at 
cat.cisti.nrc.ca.  Articles  can  be  ordered  from  CISTI  for  a  fee. 

NTIS  is  an  agency  for  the  U.S.  Department  of  Commerce’s  Technology  Administration.  It  is  the 
official  source  for  government  sponsored  U.S.  and  worldwide  scientific,  technical,  engineering  and 
business  related  information.  The  400,000  article  database  can  be  searched  for  free  at  the 
www.ntis.gov.  Articles  can  be  purchased  from  NTIS  at  costs  depending  on  the  length  of  the  article. 

IEEE  Xplore  is  the  database  for  articles  published  by  the  Institute  of  Electrical  and  Electronics 
Engineers.  It  is  searchable  online  at  ieeexplore.ieee.org.  Simple  searches  and  abstracts  are 
available  to  the  general  public  free  of  charge  from  while  more  complex  searches  and  full-text 
versions  of  articles  are  available  only  for  a  fee. 

HFES  is  a  database  of  articles  published  by  the  Human  Factors  and  Ergonomics  Society  including 
Human  Factors,  Ergonomics  in  Design,  and  the  HFES  conference  proceedings.  Searches  are 
available  for  free,  while  most  articles  and  contents  of  the  proceedings  are  available  only  for  a  fee. 
This  database  is  located  online  at  www.hfes.org/Web/. 

PubMed  is  a  publicly  available,  free  service  of  the  U.S.  National  Library  of  Medicine  that  includes 
over  16  million  citations  from  MEDLINE  and  other  life  science  journals  for  search  of  biomedical 
articles  back  to  the  1950s.  PubMed  includes  links  to  frill  text  articles  that  can  be  purchased  directly 
from  the  journal  source.  Pubmed  can  be  accessed  online  at  www.pubmed.gov. 

DRDC  Defence  Research  Reports  is  a  database  of  scientific  and  technical  research  produced  over 
the  past  6-  years  by  and  for  the  Defence  Research  &  Development  Canada.  It  is  available  online  at 
pubs.drdc-rddc.gc.ca/pubdocs/pcow  l_e.html. 

The  World  Wide  Web  was  searched  using  search  engines  such  as  Google  Scholar 
(scholar.google.com)  and  Google  (www.google.ca). 

2.3  Search  Strategy  and  Selection  of  Articles 

Initial  searches  were  performed  using  each  primary  keyword  individually.  If  the  search  resulted  in 
too  many  records  or  irrelevant  titles,  the  related  keywords  were  used.  Search  terms  from  the 
primary  column  were  combined  with  terms  in  the  related  keyword  column,  or  terms  from  within 
the  related  keyword  column  were  combined  together  whenever  the  searches  resulted  in  too  many 
records.  Upon  examination  of  the  literature,  some  additional  keywords  were  identified  and  used 
when  further  information  was  desired  (e.g.,  names  of  specific  heuristics  or  information  aggregation 
theories,  author’s  names). 

STINET  proved  to  be  the  most  effective  database  for  locating  papers  related  to  Combat 
Identification.  NTIS  and  CISTI  consisted  primarily  of  a  smaller  subset  of  repeated  titles  found 
within  the  STINET  database.  Google  Scholar  located  the  greatest  number  of  articles  directly 
related  to  models  of  information  aggregation,  particularly  in  the  fields  of  Economics  and  Political 
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Science.  IEEE  Xplore  also  revealed  papers  related  to  information  aggregation.  However,  these 
records  focused  mainly  on  data  fusion,  which  is  beyond  the  scope  of  this  review  because  of  our 
focus  on  the  human  decision  maker.  PsycINFO,  PsycARTICLES,  PubMed,  and  HFES  were 
particularly  useful  for  finding  papers  related  to  cue -based  decision  making  (naturalistic  decision 
making).  Articles  providing  military  application  of  this  form  of  decision  making  were  found  using 
STINET.  General  decision-making  literature  was  located  through  Google  Scholar  as  well  as 
PsycINFO  and  PsycARTICLES. 

2.4  Review  of  Articles 

Approximately  150  relevant  titles  were  obtained  during  the  keyword  searches.  Review  of  the 
abstracts  reduced  this  number  to  about  100.  After  reviewing  nearly  30  articles  in  detail,  the  themes 
of  the  review  became  more  apparent  allowing  for  more  specific  searches  and  selection  of  papers. 
Articles  were  subdivided  into  primary  and  secondary  references.  Articles  were  defined  as  primary 
if  their  content  was  particularly  relevant  to  a  specific  topic  covered  by  this  review  (e.g.,  method  of 
information  aggregation,  naturalistic  decision  making  in  a  military  context).  Each  of  the  64 
primary  references  were  cited  at  least  once  in  the  body  of  this  review.  Secondary  references  had 
some  relevance  to  the  themes  of  this  review  but  were  not  referenced  within  the  literature  review 
either  due  to  minimal  applicability  to  combat  identification,  the  human  decision  maker,  or  because 
of  repetition  in  topics. 
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3  Conceptualization  of  Combat 
Identification 


In  order  to  add  context  to  this  review  of  human  information  aggregation  literature,  we  begin  with  a 
discussion  of  combat  identification.  We  will  then  assess  how  a  combat  identification  judgment 
might  fit  within  the  greater  decision-making  literature.  This  will  assist  us  in  later  evaluating  the 
likely  use  of  analytic  or  intuitive  approaches  to  combining  information  in  the  battlefield. 

In  a  land  force  context,  LAV  III  commanders,  and  mounted  and  dismounted  soldiers  must  be  able 
to  form  a  rapid  identification  of  detected  entities  (e.g.,  friendly,  neutral,  hostile)  in  the  shared 
battle-space,  in  order  for  appropriate  actions  to  be  taken  (e.g.,  ignore,  shoot,  hide)  in  a  timely 
manner.  This  process  is  referred  to  as  combat  identification  (CID). 

CID  is  necessary  in  a  variety  of  circumstances.  For  example,  a  CF  (  Canadian  Forces)  unit  may 
need  to  enter  an  Afghan  village  to  locate  a  suspected  Taliban  leader.  Within  this  environment  they 
will  encounter  many  different  people,  some  of  whom  will  be  neutral  (e.g.,  farmers,  children), 
others  who  are  considered  friends  (e.g.,  coalition  forces),  and  in  addition,  there  may  be  hostile 
individuals.  As  each  person  is  encountered  in  the  village,  soldiers  must  rapidly  judge  which 
category  they  are  most  likely  to  fit.  To  make  this  judgment,  cues  taken  from  the  person  (e.g., 
appearance,  presence  of  a  weapon,  actions)  and  the  context  (e.g.,  person  is  located  in  a  known 
enemy  home)  must  all  be  considered,  while  at  the  same  time  combining  it  with  previous  knowledge 
and  expectations.  In  this  example,  identification  goes  a  step  further  requiring  a  judgment  on 
whether  an  individual  is  the  specific  person  being  sought.  In  another  example,  a  LAV  III  may  be 
manoeuvring  through  an  open  area  of  terrain  when  movement  is  sighted  in  the  distance.  If  it  is 
another  vehicle.  Identification  Friend  or  Foe  (IFF)  technology  may  be  used  to  interrogate 
transponders  found  on  friendly  vehicles,  which  will  give  an  identification  response.  However,  this 
technology  is  not  always  available  or  100%  effective  (Haspert,  2000).  If  the  entity  is  a  person  or 
group  of  people,  Budd  light  technology  (i.e.  pulsed  infra-red  beacons)  and  reflective  gear  may  be 
useful  in  assisting  with  identification  (i.e.  determining  if  they  are  Blue  forces  or  not).  However, 
enemies  may  use  similar  gear  in  order  to  cause  deception.  Therefore,  when  possible,  and  when 
enough  time  is  deemed  available,  other  cues  must  be  utilized.  For  example,  the  commander  may 
receive  communication  from  the  gunner  who  is  able  to  offer  enhanced  visual  information  about  the 
entity  (e.g.,  insignia,  type  of  weaponry)  or  radio  contact  may  be  made  with  other  units  within  the 
area  who  are  closer  to  the  source. 

As  is  apparent  with  the  above  examples,  the  ability  to  effectively  combine  information  is  critical  to 
CID.  Not  only  must  data-driven  information  (e.g.,  visual  cues,  sensors  and  communication)  be 
aggregated,  but  it  must  also  be  combined  with  cognitively  driven  elements  of  information  (e.g., 
expectations,  beliefs,  knowledge  from  previous  experiences).  In  addition,  there  may  be  multiple 
cues  from  a  single  source  (e.g.,  different  visual  cues),  or  multiple  sources  providing  single  cues  or 
judgments  (e.g.,  IFF  states  friend  or  unknown,  Budd  light  is  present  or  absent).  Historically,  vision 
was  the  primary  information  source  used  to  identify  a  detected  entity.  However,  with  present 
weaponry  easily  reaching  distances  greater  than  that  of  the  line  of  sight,  external  sensors  and 
information  sources  must  be  relied  upon  (Haspert,  2000).  The  complexity  of  the  battlefield  where 
suicide  bombers  use  deception  to  blend  in  with  crowds  and  guns  may  be  mistaken  for  hoes  or  rakes 
held  by  farmers,  requirements  for  greater  use  of  communication  and  a  continuously  updated 
situational  awareness  have  arose.  The  difficulty  with  creating  an  ID  sensor  that  is  100%  accurate  at 
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all  times,  in  all  situations  (various  weather  conditions,  terrain  and  time  of  day)  causes  the  need  to 
use  numerous  cues.  Failure  to  utilize  this  information  appropriately  can  result  in  a 
misidentification,  which  can  be  very  costly,  resulting  in  fratricide  (identifying  a  friend  as  hostile), 
neutricide  (identifying  a  neutral  entity  as  hostile)  or  suicide  (identifying  a  hostile  target  as  a  friend) 
(Karsh,  Walrath,  Swoboda,  &  Pillalamarri,  1995). 

Several  models  have  been  developed  to  explain  the  information  flow  and  decision  processes  of 
combat  identification. 

3.1  Models  of  Combat  Identification 

Before  describing  the  models  of  combat  identification,  it  is  necessary  to  define  some  CID-related 
terms  that  might  otherwise  be  confused,  since  we  have  observed  that  they  are  not  used  consistently 
in  the  literature.  The  terms  “detection”,  “identification”,  “classification”  and  “recognition”  have 
unique  meanings  in  a  military  context,  and  will  therefore  be  treated  as  such  in  this  review.  In  the 
army,  DCRI  (Detect,  Classify,  Recognize  and  Identify)  tasks  are  considered  separate  processes 
(Self,  2005): 

•  Detect.  A  vehicle,  person  or  structure  of  possible  military  interest  is  noticed.  The 
military  observer  takes  action  to  search  for  further  information. 

•  Classify.  The  object  is  distinguished  by  class,  such  as  wheeled  or  tracked  vehicle, 
animal  or  human. 

•  Recognize.  The  object  is  distinguished  by  category,  such  as  tank  or  personnel  carrier 
in  the  tracked  vehicle  class.  If  the  object  is  human,  elements  of  the  person,  such  as 
lack  or  presence  of  equipment,  head-gear,  or  posture  are  used  to  determine  if  the 
person  is  of  military  interest. 

•  Identify.  The  object  is  distinguished  by  model  (e.g.,  4  door  sedan  if  a  vehicle)  and  the 
force  allegiance  (friend,  foe,  etc)  is  determined  (but  not  confirmed).  If  the  object  is 
human,  elements  of  the  person,  such  as  clothing,  equipment,  posture  and/or  gender  are 
used  to  determine  if  the  person  is  armed  or  potentially  combatant. 

Thus,  each  successive  term  implies  greater  refinement  in  the  level  of  detail  extracted  from  the 
target  under  consideration. 

Feature  identification  has  been  proposed  as  an  additional  stage  related  to  the  discrimination  of 
vehicles  based  on  their  make  and  model,  or  between  different  types  of  weapons  and  uniforms  used 
by  people  (Self,  2005).  For  our  purposes,  the  words  “identification”,  ‘identity”  and  “identify”  will 
be  used  to  refer  to  the  determination  of  the  allegiance  of  the  detected  entity  (i.e.  friend,  neutral, 
enemy  in  accordance  with  formal  NATO  doctrine  of  “standard  identification”).  Whenever  it 
appears  that  reviewed  literature  intends  to  portray  the  meaning  of  one  of  the  above  terms,  but  is 
using  another  word,  it  will  be  changed  for  consistency. 

According  to  the  U.S.  Defense  Science  Board  (1996),  the  CID  process  starts  after  receiving 
information  about  a  detected  entity  from  both  direct  (organic  to  the  soldier)  and  indirect  (external, 
off-board)  sources.  Each  source  of  information  is  associated  with  a  probability  that  describes  the 
likelihood  of  it  providing  the  correct  identity  of  the  entity.  For  example,  Geiselman  and  Samet 
(1980)  have  recommended  use  of  certainty  levels  for  intelligence  briefings  whenever  possible. 
Humans  and/or  computers  fuse  the  information  together  to  result  in  a  statistical  indication  of  the 
likely  identity  of  the  entity  (U.S.  Defense  Science  Board,  1996).  Given  this  identity  and  its 
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certainty  level,  decision  makers  utilize  the  Rules  of  Engagement  (ROE)  to  determine  an 
appropriate  action.  This  model  highlights  the  use  of  probabilities  or  certainty  levels  when 
weighing  information  and  when  choosing  an  action.  For  example,  in  the  earlier  example  regarding 
the  search  for  a  Taliban  leader,  the  cue  that  someone  is  found  in  a  known  enemy  home  may  be 
more  important  than  the  presence  of  a  weapon,  due  to  the  prevalence  of  arms  amongst  this  culture. 
One  limitation  to  this  article  was  that  it  failed  to  describe  how  soldiers  might  carry  out  the  process 
of  weighing  and  combining  information. 

Allegretti  (2000)  offers  an  expanded  model  of  CID  that  describes  three  components  involved: 
target  identification  (TI),  situational  awareness  (SA),  and  specific  tactics,  techniques  and 
procedures  (TTP).  TI  involves  making  distinctions  at  different  levels  of  resolution  for  the 
detection,  classification,  recognition,  identification  and  feature  identification  (referred  to  as 
characterization  in  Allegretti’s  paper)  of  the  entity.  SA,  on  the  other  hand,  is  described  as  the 
knowledge  of  one’s  own  location  and  the  location  of  others.  It  is  necessary  to  note  that  this 
definition  of  SA  differs  from  the  more  commonly  cited  three-level  model  of  SA  described  by 
Endsley  (1995),  as  will  be  discussed  shortly.  Allegretti  (2000)  suggests  that  position  location 
displays  will  be  used  to  assist  in  forming  an  accurate  SA.  TTPs  are  tools  and  rules  available  to 
maximize  the  use  of  different  resources  (e.g.,  sensors).  The  model  described  by  Allegretti  (2000) 
suggests  that  TTP  (as  well  as  available  military  intelligence)  influences  both  SA  and  TI,  which 
together  interact  through  an  information  exchange  to  facilitate  a  combat  identification.  Although 
this  model  provides  more  detail  about  the  components  of  CID,  it  is  still  inadequate  to  describe  how 
CID  decision  makers  use  information. 

Dean,  Vincent,  Mistry,  Hynd,  and  Syms  (2005)  of  the  U.K.  Defence  Science  Technology 
Laboratory  are  authors  of  a  draft  paper  we  have  identified  as  containing  the  most  comprehensive 
model  of  CID  processes.  Dean  et  al.  (2005)  describe  the  information  flow  from  the  entity  to  the 
CID  decision  maker  (Figure  1).  They  also  provide  a  conceptual  model  called  the  Integrative 
Combat  Identification  Entity  Relationship  (INCIDER)  to  describe  how  a  CID  is  made.  Once  an 
entity  is  detected,  information  about  the  entity  is  acquired  from  multiple  sources.  A  fusion  process 
then  aggregates  the  available  information,  which  is  further  combined  with  memory  and  other 
reports  related  to  the  battlefield  situation.  Similar  to  the  model  described  by  Allegretti  (2000),  the 
aggregated  information  about  the  identity  of  the  entity  is  combined  with  situational  awareness 
during  the  decision  process.  Contrary  to  the  definition  of  SA  used  by  Allegretti  (2000),  Dean  et  al. 
(2005)  use  Endsley’s  (1995)  classic  three-level  model  of  SA,  which  is  as  follows: 

•  Level  1  SA  -  Perception  of  cues  and  elements  related  to  the  current  situation. 

•  Level  2  SA  -  Comprehension  of  the  current  situation  by  integrating  and  inteipreting 
the  perceived  information. 

•  Level  3  SA  -  Projection  of  understanding  of  current  events  into  the  near  future. 

Once  situational  awareness  is  achieved  and  the  information  has  been  integrated,  answers  are 
determined  for  the  following  categories  of  decisions: 

•  Detection  -  Is  the  entity  really  a  target? 

•  Classification  -  What  class  or  type  of  target  is  it? 

•  Identification  -  Is  it  friendly,  neutral  or  hostile? 

•  Action  -  What  action  is  required  (destroy,  report  or  avoid)? 
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HUMAN  SYSTEMS 


Figure  1.  Information  flow  during  a  combat  identification 
(recreation  of  Dean  et  al.,  2005,  p,  22) 

The  INCIDER  model  (Figure  2)  developed  by  Dean  et  al.  (2005)  expands  on  the  concept  of 
information  flow  to  show  the  relationship  between  different  factors  and  the  decision  making 
process.  It  shows  that  once  an  entity  is  detected,  information  acquired  through  sensors  and 
situation  awareness  is  fused  to  result  in  an  identification  associated  with  a  level  of  certainty 
(confidence).  A  decision  is  made  regarding  the  acceptability  of  this  confidence  level.  In  other 
words,  the  decision  maker  decides  whether  a  necessary  threshold  has  been  met  in  order  to  proceed. 
If  it  is  deemed  unacceptable,  the  process  is  repeated  to  acquire  more  information  by  moving  closer 
to  the  entity  or  using  additional  sensors.  The  tentative  judgment  is  fed  back  into  the  decision 
engine  to  act  as  history  and  expectations  that  will  influence  future  iterations.  Once  the  level  of 
certainty  of  the  judgment  is  acceptable  (i.e.  the  threshold  is  met),  the  identification  is  made  and  the 
process  continues  to  further  decision-making  stages  (i.e.  choice  of  a  course  of  action).  Similar  to 
the  model  described  by  the  U.S.  Defense  Science  Board  (1996),  Dean  et  al.  (2005)  focus  on  the 
need  to  reach  a  certain  level  of  confidence  in  the  identification  before  proceeding  with  a  course  of 
action.  This  level  will  change  as  the  distance  between  entities  decreases  making  the  decision  more 
urgent  (Dean  et  al.,  2005).  In  contrast  to  the  other  models,  Dean  et  al.  (2005)  suggest  three 
potential  ways  in  which  information  and  the  confidence  (probabilities)  associated  with  it  might  be 
aggregated.  These  models  of  information  aggregation  include  Dempster-Shafer  theory,  Bayes’ 
Theorem  and  Fuzzy  Fogic,  with  particular  support  towards  use  of  Dempster-Shafer  theory. 

Despite  this  inclusion,  only  Bayes’  Theorem  will  be  discussed  in  this  review  because  Fuzzy  Fogic 
and  Dempster-Shafer  theory  are  data  fusion  models  requiring  use  of  complex  algorithms,  and  are 
therefore  beyond  the  scope  of  this  review  (the  interested  reader  is  referred  to  Dean  et  al.,  2005  for 
more  information  or  to  the  following  references  in  the  secondary  reference  list:  Choi,  1999  (Fuzzy 
Fogic);  Clemen  &  Winkler,  in  press  (Bayes  Theorem);  Sentz  &  Ferson,  2002  (Dempster-Shafer)). 


Wumansystems 


Combat  Identification:  Literature  Review 


Page  10 


Bayes’  Theorem  is  also  used  for  data  fusion,  but  research  has  suggested  that  people  may  be  able  to 
utilize  this  method  in  some  circumstances  (Budescu  &  Yu,  2006;  Sedlmeier  and  Gigerenzer,  2001). 
This  will  be  discussed  further  in  Section  5:  Models  of  Information  Aggregation. 


Human  Pre-Set 
Parameters  of 
Decision  Maker 


Situation  Awareness 


Task 

selection 

Value:  X 
at  Output 
Time:  T 


Figure  2.  INCIDER  Model  (recreation  of  Dean  et  al.,  2005,  p.  43). 

These  models  of  CID  provide  a  framework  for  understanding  the  way  information  is  used  during  a 
combat  identification.  Information  must  be  acquired,  it  needs  to  be  combined,  and  then  a  decision 
must  be  made  about  whether  more  information  is  needed,  or  whether  a  judgment  can  be  formed. 
This  type  of  judgment  is  similar  to  many  everyday  decisions.  For  example,  multiple  pieces  of 
information  are  often  used  when  choosing  to  eat  at  a  specific  restaurant,  such  as  opinions  from 
friends,  public  restaurant  reviews,  knowledge  about  ones’  personal  budget  and  the  cost  of  eating  at 
the  restaurant.  People  may  also  combine  estimates  from  different  people  or  sources  of  information 
about  the  seasonal  temperature  in  a  given  location  so  that  choices  can  be  made  about  what  to  pack 
for  a  holiday.  Each  of  these  pieces  of  information  may  be  weighted  differently  when  making  these 
judgments.  Despite  the  similarities,  CID  has  two  crucial  differences  from  such  everyday  decisions, 
first  is  the  environment  where  it  takes  place,  and  second,  is  the  seriousness  of  the  consequences  of 
making  an  error.  Some  of  the  factors  affecting  CID  decisions  described  by  Dean  et  al.  (2005)  are 
outlined  in  the  next  section. 
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3.2  Factors  affecting  combat  identification 

Unlike  most  everyday  decisions,  CID  decisions  are  made  in  uncertain  environments  where  timing 
is  critical.  To  be  effective,  the  CF  needs  to  execute  their  OODA  loop  (Observe,  Orient,  Decide  and 
Act)  faster  than  the  enemy  can.  They  need  to  have  a  better  understanding  of  the  situation  so  that 
they  can  identify  enemies  faster  and  more  accurately  allowing  them  to  take  action  quicker  (Azuma, 
Daily  &  Furmanski,  2006).  To  do  this,  information  must  be  obtained  and  combined  effectively. 

A  variety  of  situational  factors  can  affect  the  information  available  to  CID  decision  makers.  For 
example,  the  terrain  difficulty,  time  of  day  (day  or  night),  and  weather  conditions  can  affect  the 
ease  with  which  people  acquire  information  about  the  detected  entity  (Dean  et  al.,  2005).  Even  the 
relief,  or  shape  of  the  ground  can  affect  decision  makers’  line-of-sight  causing  difficulty  in  forming 
an  ID.  Communication  ability  can  also  enhance  or  degrade  CID.  If  messages  are  transmitted  too 
slowly  or  contain  errors,  CID  decision  makers  may  lose  valuable  information,  resulting  in 
uncertainty  (Dean  et  al.,  2005). 

Dean  et  al.  (2005)  identified  two  categories  of  human  parameters  that  affect  CID:  human  pre-set 
parameters  and  human  variable  parameters.  Human  pre-set  parameters  of  the  decision  maker 
include  personality,  beliefs  (culture),  experience,  trust  and  preferences.  These  factors  can  influence 
the  formation  of  situational  awareness,  how  information  is  aggregated  and  how  decisions  are 
formed.  Human  variable  parameters,  which  can  also  influence  the  effectiveness  of  forming  a 
decision,  consist  of  factors  that  arise  because  of  the  intensity  of  the  situation  such  as  fear,  stress  and 
fatigue  (Dean  et  al.,  2005). 

A  factor  that  affects  how  information  is  considered  and  used  for  forming  judgments  related  to  CID 
is  the  level  of  command.  According  to  Dean  et  al.  (2005)  there  are  five  levels  of  command: 

•  Grand  Strategic  (e.g.,  chiefs  of  state,  political  influences); 

•  Military  Strategic  (force  deployment); 

•  Operational  (commander,  training); 

•  Tactical  (land,  air,  waterspace  management,  C2  arrangements,  planning  phases); 

•  Close  Tactical  (those  directly  involved  in  combat). 

Each  level  of  command  has  different  pieces  of  information  that  make  up  the  battlefield  scenario. 
Errors  made  by  those  at  higher  levels  of  command  ultimately  affect  those  at  the  lower  levels.  At 
the  close  tactical  level,  a  single  commander  may  need  to  make  the  CID  decision  by  rapidly 
aggregating  information.  In  contrast,  those  at  the  grand  strategic  level  may  be  involved  in  debate 
and  discussion  over  the  funding  towards  training,  which  will  affect  the  information  received  by 
individual  soldiers.  Although  this  review  will  focus  primarily  on  individual  decision  makers  in  the 
close  tactical  level  of  command,  some  of  the  models  of  information  aggregation  will  be  useful  for 
understanding  the  processes  used  by  those  at  higher  levels. 

This  section  has  highlighted  the  complexity  associated  with  combat  identification,  and  has 
emphasized  the  importance  of  successful  information  aggregation.  To  further  our  understanding  of 
this  task,  we  will  now  review  the  decision-making  literature  with  a  focus  on  cue-based  decision¬ 
making. 
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Decision  Making  Frameworks 


The  need  to  use  multiple  cues  (information)  to  form  a  judgment,  in  this  case  about  the  identity  of  a 
battlefield  entity,  is  referred  to  in  general  terms  as  cue-based  decision-making.  There  are  several 
branches  of  decision-making  literature  that  must  be  described  in  order  to  understand  where  this 
term  fits.  Primary  among  these  are  analytic  decision  theory  and  intuitive  decision  theory. 

4.1  Analytic  Decision  Theory 

Analytic  decision  theory,  also  called  normative,  rational  or  classical  decision  making,  focuses  on 
the  selection  of  the  optimal  course  of  action  (COA)  from  all  possible  alternatives  derived  from  all 
available  information.  This  theory  basically  views  decision  making  as  a  statistical  problem  and 
therefore  makes  extensive  use  of  computations.  Analytic  theory  is  referred  to  as  prescriptive 
because  it  explains  how  decision  makers  should  make  decisions.  Klein  and  MacGregor  (1988,  p. 
14)  summarize  normative  decision  analysis  as  involving  seven  processes: 

1)  Enumerate  all  possible  COAs; 

2)  Identify  the  possible  outcomes  associated  with  each  COA; 

3)  Place  values  on  each  outcome; 

4)  For  each  COA,  assess  the  probability  that  the  associated  outcome  would  be 
experienced; 

5)  Multiply  each  outcome  by  its  associated  probability  to  obtain  an  expected  value  or 
utility; 

6)  Sum  the  expected  values  for  each  COA;  and 

7)  Choose  the  course  of  action  having  the  largest  expected  value  (i.e.  greatest  benefit). 

This  can  clearly  be  a  cognitively  demanding  process  requiring  both  time  and  effort.  Klein  (1997) 
has  suggested  that  this  kind  of  decision-making  only  occurs  when  the  decision  maker  is  a  novice, 
there  is  ample  time,  the  decision  maker  must  justify  their  choice  to  others,  the  best  option  must  be 
determined,  or  when  the  task  involves  computational  complexity.  Therefore,  in  an  operational 
context,  it  is  likely  that  this  kind  of  decision-making  is  restricted  to  those  at  high  levels  of 
command  who  are  involved  in  the  development  of  the  rules  of  engagement,  training  methods,  plans 
and  policies,  rather  than  those  involved  in  time-pressured  CID. 

4.2  Intuitive  Decision  Theory 

The  second  category  of  decision  theory,  intuitive  decision-making,  is  referred  to  as  descriptive 
because  it  purports  to  describe  what  people  actually  do  when  making  a  decision.  It  is  made  up  of 
two  main  subclasses:  Behavioural  Decision  Theory  (BDT)  and  Naturalistic  Decision  Making 
(NDM).  These  subclasses  focus  on  satisficing,  therefore  achieving  an  adequate  decision,  rather 
than  an  optimal  one,  in  a  real-life  environment. 
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4.2.1  Behavioural  Decision  Theory  (BDT) 

BDT,  also  referred  to  as  bounded  rationality,  was  developed  after  analytic  decision  theories  failed 
to  recognize  that  people  are  bound  by  their  cognitive  abilities,  causing  them  to  stray  from 
normative  decision  strategies  (Einhorn  &  Hogarth,  1981;  Gigerenzer,  2004).  In  real-life  situations, 
people  often  use  heuristics  and  cognitive  biases  because  they  do  not  have  the  opportunity  to 
systematically  process  the  information  due  to  cognitive  limitations.  For  example,  heuristics  may  be 
used  to  reduce  the  amount  of  information  available  for  making  a  judgment  in  order  to  avoid 
cognitive  overload,  particularly  when  under  pressure  (Einhorn  &  Hogarth,  1981;  Bazerman,  2001). 
Theories  of  bounded  rationality  often  refer  to  such  heuristics  and  biases  as  weaknesses  because 
they  detract  from  the  optimal  decision  (Lipshitz,  Klein,  Orasanu,  &  Salas,  2001b).  However,  in 
some  circumstances,  heuristics  offer  solutions  indistinguishable  from  normative  strategies,  but  with 
decreased  effort  and  time  (Gigerenzer,  2004).  Tversky  and  Kahneman  are  well  known  for  their 
classification  of  heuristics  and  biases  used  by  people  when  they  are  under  pressure  or  making 
judgments  in  uncertain  situations  (e.g.,  Tvsersky  &  Kahneman,  1982a,b,c).  In  addition,  Gigerenzer 
has  developed  the  Adaptive  Toolbox  to  describe  the  variety  of  fast  and  frugal  heuristics  people  use 
to  adapt  to  their  situation  (e.g.,  Gigerenzer,  2004).  Relevant  heuristics  and  biases  will  be  described 
when  discussing  the  different  models  of  information  aggregation  (Section  5).  This  form  of 
decision-making  has  greater  relevance  for  decision  makers  forming  a  combat  identification  because 
it  focuses  on  real-life  with  an  emphasis  on  rapid  decision  making. 

4.2.2  Naturalistic  Decision  Making  (NDM) 

NDM  was  first  described  in  the  late  1980’s  following  the  development  of  BDT.  Its  focus  is  on 
understanding  how  experience  affects  decision-making  in  dynamic,  uncertain,  ill-defined,  high 
stress,  time-pressured  environments  (Lipshitz,  Klein,  Orasanu,  &  Salas,  2001a).  This  narrow  scope 
has  made  it  ideal  for  examining  specific  groups  of  people  such  as  military  commanders,  fireground 
commanders,  pilots  and  emergency  room  physicians  (e.g.,  Bonifay,  2000;  Lipshitz,  Sender, 
Omodei,  McClellan,  &  Wearing,  in  press;  Falzer,  2004,  O’Hare  &  Wiggins,  2004).  NDM  research 
suggests  that  in  real-life  situations,  people  rarely  make  use  of  all  available  information  and  do  not 
take  time  to  generate  or  evaluate  all  possible  alternatives.  Instead,  decision  makers  rely  on  a  fast 
and  frugal  use  of  cues  from  the  environment  and  previous  experiences  to  develop  an  adequate 
course  of  action.  It  is  this  decision  making  approach  that  can  be  considered  synonymous  with  cue- 
based  decision-making,  one  of  the  focuses  of  this  review. 

Lipshitz  et  al.  (2001a)  present  five  essential  characteristics  of  NDM: 

1)  Proficient  decision  makers.  Decision  makers  are  experts  in  the  domain  being  studied. 

2)  Situation-action  matching  rules.  Decision  makers  match  situations  to  a  COA  instead 
of  generating  and  comparing  all  possible  choices.  Matching  may  be  an  analytic 
process,  but  more  likely  involves  intuitive  pattern  matching. 

3)  Context-bound  informal  modelling.  Recognizing  patterns  and  cues  requires  context 
specific  expert  knowledge. 

4)  Process  orientation.  NDM  focuses  on  the  cognitive  processes  of  expert  decision 
makers,  not  on  predicting  the  options  they  will  choose. 

5)  Empirical-based  prescription.  Determining  what  decision  makers  should  do 
(prescription)  is  derived  based  on  observations  and  reports  of  expert  performance,  not 
on  the  normative  (optimal)  rules. 
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These  characteristics  have  been  utilized  in  several  different  models  of  NDM.  Two  of  these  models 
will  be  discussed  here  for  their  relevance  to  cue-based  decision-making,  including  the  Lens  Model 
and  Recognition-Primed  Decision  (RPD)  Model. 

4. 2. 2. 1  Lens  Model 

The  Brunswikian  Lens  Model  (Figure  3)  is  a  model  of  cue  integration  and  belief  formation  (Horrey 
&  Klein,  2001).  A  belief  can  lead  to  three  outputs,  including  the  choice  of  an  action  (e.g.,  choose 
to  initiate  weapons  due  to  the  belief  that  the  entity  is  hostile),  allocation  of  resources  (e.g.,  seek 
more  information  from  a  second  look,  from  superiors  or  technology  to  resolve  uncertainty  about 
the  identity)  and  assessment  of  the  degree  of  confidence  in  the  diagnosis.  The  Lens  Model 
addresses  cue-based  decision  making  by  using  linear  models  that  deal  with  how  people  subjectively 
weigh  information.  Particularly,  it  describes  the  relationship  between  what  people  perceive  about 
cues  in  the  environment  to  the  true  value  of  the  cues.  Klein  and  MacGregor  (1988)  describe  this 
model  as  a  way  to  distinguish  between: 

“an  object  that  is  represented  by  a  collection  of  cues,  and  the  psychological  representation 
of  that  object  in  terms  of  a  judgmental  policy  based  on  those  cues.  Like  an  optical  lens,  the 
environment  is  portrayed  in  terms  of  a  set  of  information  components  with  predictive 
importance  expressed  in  terms  of  validity  coefficients.  ’’  (p.  11) 

In  other  words,  in  a  CID  context,  a  person  in  the  environment  is  made  up  of  a  series  of  cues  (e.g., 
helmet,  weapon,  etc)  that  have  different  levels  of  predictive  power  (certainty)  for  determining  the 
person’s  identity.  The  decision  maker  has  a  “policy”  for  using  those  cues  developed  through 
training  and  experience.  The  Lens  Model  is  used  to  evaluate  whether  that  policy  lines  up  with  the 
true  meaning  of  those  cues.  It  can  be  used  for  assessing  individual  judgment  formation  in  specific 
situations  (Bisantz,  Gay,  Phipps,  Walker,  Kirlik,  &  Fisk,  1997).  The  closer  people  can  match 
(calibrate)  their  subjective  weightings  of  the  cues  to  the  objective  weightings,  the  better  their 
overall  situational  awareness  will  be,  which  will  ultimately  affect  the  accuracy  of  their  judgment 
(Wickens  et  al.,  1999).  This  weighing  process  will  be  discussed  further  in  Section  5.4. 
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Figure  3.  Model  of  cue  integration  and  belief  formation  based  on  the  Brunswikian 
Lens  Model  (recreation  of  Kirlik,  1995  as  presented  in  Horrey  and  Wickens,  2001,  p. 

4). 

Horrey  and  Wickens  (2001)  used  the  Lens  Model  in  two  experiments  to  determine  how  automation 
should  be  used  for  CID.  The  experiments  involved  battlefield  simulations  where  participants  had 
to  use  large  amounts  of  information  presented  on  map  displays  in  a  time  pressured  situation. 
Participants  had  to  assess  the  threat  level  of  enemy  units  located  on  the  display  based  on  evidence 
from  multiple  cues,  such  as  unit  type,  size,  and  separation  distance  from  own  unit,  and  terrain 
difficulty.  Each  cue  had  high  reliability  (confirmed  information),  medium  or  low  (unconfirmed) 
reliability.  In  experiment  1 ,  for  some  of  the  trials,  a  cueing  aid  highlighted  enemy  units  expected  to 
be  most  relevant  during  a  threat  assessment  task.  Therefore,  the  automation  was  designed  to  assist 
participants  in  gathering  relevant  information.  Participants,  assuming  the  position  of  a  battlefield 
commander,  had  to  determine  whether  there  was  a  greater  threat  to  the  east  or  west  of  their  unit 
(located  in  the  centre  of  the  map)  and  allocate  resources  accordingly.  This  required  mental 
integration  of  the  information. 

Prior  to  the  experiment,  a  linear  model  was  determined  to  weigh  each  piece  of  information  based 
on  questionnaire  results  from  independent  subject  matter  experts  (SME).  This  was  done  to 
determine  the  objective  value  of  each  cue.  In  this  example,  the  resulting  equation  was:  IVunit  = 
Xtype(90+4Xsize-  5Xdist-  14Xdiff)*  R  such  that  threat  level  (or  information  value,  IVunit)  of  the 
detected  unit  increased  as  the  size  of  the  unit  increased,  separation  distance  decreased  and  terrain 
difficulty  decreased.  This  equation  was  used  to  determine  the  optimal  allocation  of  defensive 
resources,  which  was  compared  to  the  actual  performance  so  that  error  scores  could  be  calculated. 
Horrey  and  Wickens  (2001)  found  that  trials  where  enemy  units  were  automatically  cued  resulted 
in  closer  to  optimal  resource  allocation,  and  faster  response  times,  than  in  trials  without  the 
automation.  This  result  suggests  that  people  benefit  from  a  system  that  assists  with  gathering 
appropriate  information.  A  memory  probe  test  was  used  to  determine  how  deeply  people  processed 
(weighed)  different  information,  which  found  that  unit  size  was  the  most  important  predictor  for 
how  participants  determined  the  threat  level. 
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In  experiment  2,  an  automatic  diagnostic  aid  was  used  to  assist  participants  in  aggregating  the 
information,  by  suggesting  which  of  the  east  or  west  side  should  receive  a  greater  allocation  of 
resources.  Results  revealed  superior  performance  compared  to  the  absence  of  automation  (Horrey 
&  Wickens,  2001).  In  fact,  performance  was  also  superior  to  use  of  the  information  gathering  aid 
in  experiment  one.  Therefore,  participants  received  additional  benefits  when  the  information 
aggregation  was  performed  for  them.  One  disadvantage  to  this  automation  was  that  participants 
failed  to  process  the  cues  as  deeply  when  the  information  aggregate  was  already  available  to  them, 
as  revealed  through  memory  probes.  This  suggests  that  during  an  automation  failure,  people  might 
not  be  as  effective  at  recognizing  the  problem.  This  experiment  could  not  confirm  this  possibility 
because  of  using  only  one  trial  where  the  automation  was  unreliable. 

These  experiments  were  described  to  show  that  the  Lens  Model  offers  a  way  to  evaluate 
performance  and  determine  how  people  weigh  cues  during  judgment  formation. 

4. 2.2.2  Recognition-Primed  Decision  (RPD)  Model 

RPD  is  more  directly  related  to  the  characteristics  of  NDM,  and  provides  a  relevant  way  to 
consider  CID  decision  makers.  Klein  (1997)  states  that  domain-specific  experts  making  a  decision 
in  a  high  stress  field  setting  where  a  reasonable,  but  not  necessarily  optimal  solution  is  required  are 
likely  to  use  Recognition-Primed  Decision  making  strategies.  Unlike  other  decision  making 
theories  that  rely  upon  making  a  choice  between  options,  RPD  involves  assessing  a  situation  and 
judging  it  as  familiar  to  previously  experienced  events  which  are  associated  with  an  action  (Klein, 
1997).  The  classic  RPD  example  is  provided  by  interviews  from  fireground  commanders  who 
were  found  to  identify  a  course  of  action  by  sizing  up  the  situation  and  then  recognizing 
appropriate  actions  to  take  (Klein,  Calderwood,  &  Clinton-Cirocco,  1986  as  cited  in  Klein,  1997). 
Fireground  commanders  reported  imagining  the  outcome  by  using  mental  simulation  to  decide  if 
the  COA  was  appropriate.  If  the  simulation  revealed  problems  and  there  was  sufficient  time, 
another  COA  was  considered  (Klein,  1997).  Therefore,  decision  makers  using  an  RPD  strategy 
evaluate  options  serially,  instead  of  concurrently  as  in  normative  decision  making  (Klein  & 
Calderwood,  1996). 

There  are  three  variations  of  the  RPD  model  (Figure  4  on  next  page).  The  first  and  simplest  of  the 
three  describes  cases  when  an  experienced  decision  maker  assesses  the  situation  and  then  uses 
existing  prototypes  to  identify  a  single  COA  (Klein  &  MacGregor,  1998).  A  prototype  or  category 
may  be  a  mental  representation  of  a  pairing  between  a  situation  and  an  action.  It  is  sometimes 
described  as  a  schema,  script  or  exemplar,  which  is  developed  over  time  with  experience  (Perrin  et 
al.,  2001).  This  variation  involves  sizing  up  the  situation,  forming  expectancies  about  what  will 
happen  next,  determining  which  cues  are  most  relevant,  recognizing  the  goals  reasonable  to  pursue 
and  recognizing  a  reaction  to  apply  in  the  situation  so  that  it  can  be  implemented  (Klein,  1997). 

The  second  variation  occurs  when  the  situation  is  not  clear  due  to  uncertainty.  Decision  makers 
use  feature  matching  where  they  identify  relevant  aspects  of  the  situation  to  assist  with  categorizing 
it,  and  story  building  where  they  use  mental  simulation  to  determine  the  events  that  may  have  led 
to  the  observed  situation  (Klein,  1997).  The  final  variation  is  the  most  complex.  It  occurs  in  cases 
similar  to  that  described  by  the  fireground  commanders  and  involves  evaluation  of  a  COA  without 
comparing  it  to  other  alternatives.  This  occurs  by  mentally  simulating  the  outcome  to  determine 
whether  it  will  work  out  as  expected.  If  the  COA  is  deemed  inappropriate,  a  new  COA  will  be 
evaluated. 

Overall,  RPD  is  made  up  of  several  components  including  situation  assessment,  pattern  matching 
and  mental  simulation.  Situation  assessment,  the  process  of  “sizing-up”  a  situation,  is  the  most 
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relevant  for  our  purposes  because  it  is  in  essence  a  model  of  information  aggregation.  Klein  and 
Calderwood  ( 1 996)  identify  four  types  of  information  relevant  to  situation  assessment  in  an  RPD 
context: 

1)  Plausible  goals.  These  are  goals  that  the  decision  maker  wants  to  achieve  in  the  given 
context. 

2)  Critical  cues  and  causal  factors.  Experts  prioritise  cues  so  that  they  do  not  focus  their 
attention  where  it  is  not  needed.  In  contrast,  novices,  who  are  excluded  from  the  RPD 
model,  may  be  overloaded  by  the  variety  of  cues  available  in  the  environment.  Experts 
have  learned  that  specific  cues  have  causal  implications  for  the  situation. 

3)  Expectancies.  These  are  ideas  about  what  is  likely  to  happen  in  a  given  situation, 
which  will  be  useful  when  determining  the  time  course  of  events.  Inexperienced 
decision  makers  will  not  have  developed  expectancies  to  the  same  degree  as 
experienced  decision  makers,  and  will  not  be  able  to  evaluate  their  situation 
assessment  by  comparing  it  to  their  expectancies. 

4)  Typical  actions.  Situations  familiar  to  a  decision  maker  should  elicit  a  set  of  typical 
actions  useful  for  a  response. 

These  types  of  information  can  be  described  in  terms  of  a  CID  decision.  A  plausible  goal  would  be 
the  desire  to  identify  the  detected  entity.  Experts  evaluate  only  the  most  relevant  cues,  which  will 
be  influenced  by  their  expectations  about  the  situation.  Once  an  understanding  is  reached  about  the 
situation,  typical  actions  to  respond  to  the  entity  will  be  elicited.  As  the  various  cues  and 
expectancies  are  aggregated,  a  coherent  situation  assessment  forms,  which  will  impact  overall 
situational  awareness  and  all  subsequent  decisions  and  performance  (Horrey  &  Wickens,  2001). 
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A.  Automatic  RPD 


B.  Verified  RPD 


C.  Serial  RPD 


Figure  4.  Models  of  RPD  (recreation  from  Klein  &  MacGregor,  1998,  p.  18). 

Pattern  matching  and  mental  simulation,  in  contrast,  are  involved  in  the  latter  stages  of  the  decision 
process  specific  to  selection  of  a  COA.  O’Hare  and  Wiggins  (2004)  studied  pattern  matching 
abilities  of  pilots  to  determine  how  they  used  previous  experience  (i.e.  knowledge  of  similar  cases) 
to  match  actions  to  emergency  situations.  Pilots  were  able  to  match  previous  cases  and  scenarios 
found  in  their  long-term  memory  (LTM)  to  their  assessment  of  the  current  situation  (i.e.  their 
information  aggregate)  in  order  to  identify  a  COA.  Therefore,  pilots  compared  and  matched 
previous  situations  with  an  aggregate  of  the  current  case.  This  process  suggests  that  people  must 
form  an  aggregate  sufficiently  coherent  that  it  can  be  related  to  features  of  past  examples. 

Therefore,  although  RPD  does  not  explicitly  discuss  models  of  information  aggregation,  it  is 
apparent  that  the  situation  assessment  results  in  a  combination  of  cues.  One  potential  problem  that 
can  arise  with  pattern  matching  is  a  false  positive.  This  occurs  when  people  assume  a  situation  is 
similar  enough  to  a  previously  experienced  one  that  they  utilize  the  associated  action,  when  in 
reality  the  situation  required  a  completely  new  solution.  This  is  referred  to  as  scenario  fulfilment 
and  was  tragically  observed  when  the  USS  Vincennes  shot  down  Iran  Flight-655  in  1988  partially 
because  of  false  expectations  based  on  the  similarity  of  the  situation  to  the  Iraqi  fighter  jet  attack  on 
the  USS  Stark  the  previous  year  (Craig,  Morales,  &  Oliver,  2004).  A  way  to  avoid  this  pattern- 
matching  problem  may  be  by  building  ones’  repertoire  of  experiences  so  that  there  are  more 
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examples  available  for  appropriate  matching.  Bonifay  (2000)  has  suggested  using  simulations  to 
train  in  as  many  scenarios  as  possible. 

Interestingly,  an  interview  between  T.  Lamoureux  of  Yhim&wystems ®  and  two  members  of  the  1- 
PPCL1  (First  Battalion  of  the  Princess  Patricia’s  Canadian  Light  Infantry)  who  had  recently 
returned  from  Afghanistan,  revealed  the  use  of  RPD  strategies  during  CID  activities  (Interview, 

Feb.  9,  2007,  Edmonton,  Alberta).  The  soldiers  repeatedly  stated  that  they  had  to  “feel  the 
situation”  in  order  to  identify  whether  someone  was  an  enemy,  neutral  or  friend.  One  main  cue 
associated  with  “friend”  was  the  presence  of  a  helmet.  Enemies  were  identified  as  people  holding  a 
gun  without  attempting  to  hide  it.  Assessing  the  situation  was  considered  critical  to  the  task.  For 
example,  the  interviewees  reported  noticing  when  certain  locations  (e.g.,  market)  were  busy  or  not. 
When  a  typically  crowded  location  was  not  crowded,  this  absence  of  people  provided  a  cue  that 
something  was  about  to  happen,  causing  them  to  be  on  high  alert  for  anything  out  of  the  ordinary. 
The  soldiers  also  explained  that  CID  decisions  are  made  rapidly  requiring  use  of  their  rules  of 
engagement.  They  noted  that  there  was  not  time  to  discuss  the  situation  with  those  at  higher  levels 
of  command.  However,  communication  was  common  between  people  when  time  allowed.  For 
example,  when  travelling  as  a  group  of  LAVs,  there  were  occasions  where  a  number  of  LAV 
commanders  contacted  each  other  regarding  the  identity  of  a  detected  entity.  The  limited  details 
regarding  specific  cues  used  to  classify  people  suggest  that  the  aggregation  process  or  situation 
assessment  may  be  highly  automatic  in  those  with  experience. 

4.3  Summary 

In  summary,  the  decision-making  literature  is  made  up  of  analytic  and  intuitive  processes  that  vary 
in  their  goals  (optimal  vs.  satisficing)  and  in  the  effort  required  to  reach  them  (time  consuming  vs. 
rapid).  Combat  identification  is  most  strongly  related  to  the  NDM  models,  which  focus  on 
subjective  weighting  of  cues  and  situation  assessment  necessary  for  cue-based  decision  making. 
This  framework  will  be  helpful  in  evaluating  the  usefulness  of  the  models  and  principles  associated 
with  information  aggregation. 
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5  Models  of  Information  Aggregation 


As  has  already  been  observed  in  our  review  of  CID  and  the  decision-making  frameworks, 
information  aggregation  is  an  important  aspect  of  decision-making.  It  involves  both  weighing  and 
combining  information  (cues)  that  is  selected  for  relevance  to  the  decision  in  a  preceding  process  of 
information  gathering  (Einhom  &  Hogarth,  1981).  The  information  aggregation  stage,  the  focus  of 
this  review,  involves  combining  quantitative  and/or  qualitative  information  into  a  single  output 
(e.g.,  estimate,  probability,  belief,  or  hypothesis)  used  to  make  a  judgment.  In  the  literature,  studies 
involve  the  aggregation  of  quantitative  estimates,  probabilities,  opinions,  binary  choices,  or 
qualitative  descriptions.  The  research  identified  is  often  specific  to  the  aggregation  of  information 
from  different  sources  (e.g.,  vision,  communication,  different  sensors).  However,  it  is  expected 
that  this  research  should  extrapolate  to  multiple  cues  that  come  from  the  same  source  (e.g., 
different  visual  cues). 

The  study  of  information  aggregation  is  particularly  important  given  the  limitations  associated  with 
human  information  aggregation  abilities  (Westen  &  Weinberger,  2004;  Horrey  &  Wickens,  2001). 
Westen  and  Weinberger  (2004)  suggest  that  people  are  poor  at  integrating  information  because 
they  rarely  receive  feedback  regarding  their  cue-weighing  accuracy,  therefore  limiting  their  ability 
to  refine  the  weights  that  they  use.  In  contrast,  it  has  been  suggested  that  people  are  fairly  good  at 
identifying  relevant  cues  during  the  information-gathering  phase  (Horrey  &  Wickens,  2001;  see 
Gruppen,  Wolf  &  Billi,  1991  for  an  example  where  people  are  poor  at  the  information  gathering 
phase).  People  also  perform  adequately  at  coding  the  directional  relationship  between  cues  and  the 
judgment  (e.g.,  a  person  wearing  coalition  insignia  provides  a  positive  weight  towards  the 
judgement  of  “friendly”)  (Westen  &  Weinberger,  2004;  Dawes,  1982). 

The  difficulty  associated  with  information  aggregation  has  made  it  a  target  for  development  of 
automatic  support  systems  (Horrey  &  Wickens,  2001).  Despite  the  difficulties  with  this  process,  it 
is  a  necessary  one,  not  only  from  the  perspective  of  the  practicality  in  designing  a  1 00%  accurate 
identification  sensor  as  discussed  in  Section  3,  but  also  due  to  statistical  reasons  which  will  be 
discussed  next. 

5.1  Reasons  for  using  multiple  information  sources  during  decision 
making 

“Information  pertinent  to  individual  decisions  never  exists  in  concentrated  form, 
but  solely  as  the  dispersed  bits  of  incomplete  and  frequently  contradictory 
knowledge  which  all  the  separate  individuals  possess’’  (Hayek,  1945,  p.  519  as 
cited  in  Piketty,  1999). 

It  has  long  been  recognized,  particularly  in  the  social  sciences  (i.e.  Economics,  Political  Science, 
Organizational  Behaviour),  that  aggregating  information  from  multiple  sources  can  lead  to  the  most 
informed  decision  (Yaniv,  2004;  Budescu  &  Yu,  2006,  Feddersen,  &  Pesendorfer,  1999,  Wit, 

1998).  As  stated  in  the  above  quote  by  Hayek  (1945),  information  often  only  exists  dispersed 
across  individuals  (or  information  sources).  This  may  be  because  people  have  different 
experiences  and  knowledge,  which  influences  their  interpretation  of  events  and  evidence 
(Feddersen,  &  Pesendorfer,  1999).  For  example,  a  soldier  may  have  visited  a  particular  village  in 
the  past  therefore  acquiring  knowledge  about  its  organizational  structure.  Upon  returning  with  a 
soldier  inexperienced  with  this  village,  he  will  be  able  to  offer  insight  into  the  neutrality  of  certain 
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individuals.  To  make  best  use  of  the  private  information  held  by  different  individuals,  it  is 
necessary  to  poll  them  for  their  opinion.  In  ideal  situations,  individuals  provide  genuine  responses 
based  solely  on  their  private  information  (Feddersen,  &  Pesendorfer,  1999).  A  loose  analogy 
related  to  this  use  of  dispersed  information  is  Condorcet’s  Jury  Theorem.  This  theory  was 
developed  in  the  1700’s  specific  to  collective  decision  making.  It  states  that  when  responses  are 
genuine,  the  more  people  who  vote,  the  more  informed  the  judgment  will  be.  Although  this  theory 
is  not  specific  to  individual  decision  making,  and  deals  with  aggregating  opinions  rather  than 
aggregating  ‘facts’  (which  may  include  the  opinions  of  others,  but  may  equally  include  immutable 
information  about  the  environment),  the  ideas  espoused  by  Condorcet’s  Jury  Theorem  are 
analogous  to  the  case  of  an  individual  decision  maker  using  and  aggregating  different  sources  of 
information  to  make  a  decision.  For  example,  this  concept  is  present  when  doctors  use  multiple 
tests  to  diagnose  a  patient.  Patients  are  subjected  to  numerous  tests  because  there  is  no  single 
source  capable  of  providing  all  relevant  information  needed  to  make  a  judgment.  For  the 
remainder  of  this  report,  we  treat  the  aggregation  of  opinions  as  analogous  to  the  aggregation  of 
multiple  information  sources.  The  reader  should  note  that  we  did  not  find  any  discussion  in  the 
literature  of  the  equivalency  of  these  different  aspects. 

Budescu  (2006)  as  cited  in  Budescu  and  Yu  (2006)  has  identified  four  reasons  for  considering 
multiple  sources  when  forming  a  judgment: 

1)  To  maximize  the  amount  of  information  for  a  decision; 

2)  To  reduce  the  effect  of  extreme  sources; 

3)  To  reduce  the  effect  of  unreliable  and  inaccurate  information;  and 

4)  To  make  the  decision  process  more  inclusive  and  ecologically  representative. 

Therefore,  not  only  are  multiple  sources  used  in  order  to  acquire  as  much  private  information  as 
possible,  but  it  is  also  done  to  reduce  the  effect  of  error  and  extreme  information,  which  may  be 
caused  by  biases  or  in  a  CID  context,  by  a  malfunctioning  sensor.  If  only  one  source  is  used,  then 
there  is  the  possibility  of  making  an  incorrect  judgment  because  of  choosing  to  rely  upon  an 
inaccurate  piece  of  information. 

In  addition,  increasing  the  number  of  sources  (e.g.,  visual  cues  from  the  decision  maker  and  sensor 
cues)  statistically  improves  the  level  of  confidence  in  a  judgment  (Budescu,  Rantilla,  Yu  & 

Karelitz,  2003;  Yaniv,  2004).  This  is  important  because  most  judgments  require  that  a 
predetermined  level  of  certainty  be  reached  before  forming  a  final  judgment  (Dean  et  al.,  2005; 
Todd  &  Gigerenzer,  2001).  Confidence  is  improved  because  the  amount  of  random  error  is 
reduced  as  the  number  of  sources  increase.  A  single  quantitative  estimate,  such  as  the  average 
seasonal  temperature  in  Toronto,  provided  by  an  individual  or  a  measurement  device,  will  contain 
random  error  due  to  fluctuations  in  the  source’s  performance  and  bias  caused  by  tendencies  to 
make  over  or  underestimates.  However,  assuming  that  individual  sources  have  minimal  biases,  the 
random  error  will  be  reduced  when  estimates  are  acquired  and  averaged  across  sources,  causing  the 
aggregated  estimate  to  converge  towards  the  true  value  (Einhom,  Hogarth  &  Klempner,  1977  as 
cited  in  Yaniv,  2004).  Realizing  this  statistical  truth  will  enhance  confidence  in  the  final  aggregate, 
therefore  improving  the  certainty  of  the  judgment. 

In  order  to  achieve  the  benefits  associated  with  using  multiple  sources,  it  is  necessary  that  the 
sources  be  at  least  partially  independent.  This  is  true  of  the  opinions  of  advisors  and  of  cues 
acquired  from  the  environment.  In  group  decision-making  literature,  the  necessity  of  independence 
is  rooted  in  the  assumptions  of  theories,  such  as  Condorcet’ s  Jury  Theorem  which  assumes  that 
individuals  have  some  degree  if  independence  allowing  them  to  provide  unique,  private 
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information  when  making  their  vote  (Feddersen  &  Pesendorfer,  1999;  Yaniv,  2004).  In  a  similar 
fashion,  cues  used  by  an  individual  decision  maker  (which  could  be  in  the  form  of  a  judgment  such 
as  an  IFF  sensor  reporting  a  judgment  of  friend,  or  an  environmental  cue  such  as  the  presence  of 
friendly  insignia)  need  a  degree  of  independence  so  as  not  to  waste  the  decision  makers’  effort  and 
time  in  evaluating  redundant  cues.  According  to  Yaniv  (2004),  redundancy  occurs  as  the  number 
of  people  or  sources  increases,  causing  a  lack  of  added  value  to  the  aggregate.  With  regards  to 
aggregating  opinions,  Y aniv  (2004)  suggests  that  only  3  to  6  opinions  are  required  to  reap  the 
benefits  of  counsel  because  beyond  this,  the  level  of  dependence  increases  resulting  in  “more  of  the 
same.”  In  addition,  the  gain  in  accuracy  may  be  decreased  if  advisors  are  highly  biased  or  if  their 
opinions  are  highly  correlated.  The  correlation  between  sources  increases  because  the  individuals 
may  have  similar  backgrounds  or  may  even  consult  one  another  (Yaniv,  2004).  Sources  other  than 
people  must  also  be  partially  independent  to  improve  judgments.  For  example,  a  sensor  that  is  only 
capable  of  detecting  an  object  at  the  same  distance  as  natural  vision  will  be  unable  to  provide 
additional  information  about  the  detected  entity.  Therefore,  it  will  not  benefit  the  identification 
process  and  may  slow  judgment  time  because  of  the  effort  to  gather  and  integrate  the  redundant 
information. 

5.2  Review  of  the  information  aggregation  literature 

Initial  investigation  of  the  literature  revealed  that  formal  models  using  the  terminology 
“information  aggregation”  were  primarily  located  in  the  Social  Sciences  (i.e.  Economics,  Political 
Science,  Organizational  Behaviour)  and  Computer  Science  disciplines.  The  search  was  expanded 
to  include  heuristics  and  psychological  principles  related  to  the  concept  of  combining  information 
to  achieve  a  more  comprehensive  understanding  of  this  process.  Across  this  literature  there  was 
minimal  application  to  military  operations  (with  the  exception  of  data  fusion  algorithms  in 
Computer  Science  which  are  beyond  the  scope  of  this  review,  and  some  papers  related  to  certain 
heuristics).  However,  the  models  were  able  to  provide  a  variety  of  concepts  that  may  be 
transferable  to  CID  decision-making  tasks. 

Several  themes  became  apparent  when  reviewing  the  literature: 

•  There  are  both  analytic  and  intuitive  aggregation  methods; 

•  Research  has  focused  on  the  aggregation  of  quantitative  information  (e.g.,  forecasts, 
probabilities,  estimates),  with  less  study  in  the  area  of  combining  qualitatively  different 
information  (i.e.  combining  different  kinds  of  information,  such  as  visual  cues  and 
numeric  information); 

•  Information  to  be  aggregated  may  be  cues  (raw  data)  or  opinions/judgments; 

•  Information  sources  may  be  independent  or  dependent  (redundant); 

•  The  way  information  is  presented  (simultaneous  or  sequential  ordering)  affects  how 
informative  it  is  and  how  it  is  aggregated; 

•  Group/team  decision  making  differs  from  individual  decisions  made  based  on 
information  from  a  group  (only  the  latter  is  of  interest  for  this  review); 

•  Weighing  information  from  self  differs  from  weighing  information  from  others; 

•  Outlying  information  is  treated  differently  than  consensus  information;  and 

•  Trust  levels  affect  information  aggregation. 
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Although  this  review  will  not  be  sectioned  by  these  themes  because  of  their  overlapping  principles, 
their  concepts  will  become  more  clearly  understood  throughout  the  course  of  this  discussion.  The 
review  will  begin  with  a  summary  of  formal  models  of  information  aggregation  and  models  of 
judgment  related  to  this  process.  Following  will  be  details  specific  to  the  process  of  weighing 
information,  and  finally  factors  related  to  the  presentation  of  the  information. 

5.3  Integrating  information 

Two  main  categories  of  research  related  to  information  aggregation  were  identified  during  the 
search:  aggregation  of  qualitatively  different  information  and  aggregation  of  quantitative 
information.  Each  of  these  categories  offers  theories  and  principles  regarding  how  people  may 
combine  information  when  making  decisions.  The  following  sections  will  review  these  principles, 
providing  CID  related  examples  whenever  possible.  It  is  necessary  to  note  that  the  methods 
described  will  include  aggregation  of  opinions  or  estimates  from  others  and  the  integration  of 
environmental  cues.  Although  the  latter  is  most  directly  related  to  CID,  there  may  be  situations 
where  decision  makers  will  seek  counsel  when  making  an  ID  or  will  use  a  source  that  provides  a 
judgment  (e.g.,  IFF  sensor  reports  a  judgment  of  friend)  instead  of  a  raw  cue  (e.g.,  presence  of  a 
helmet),  making  the  former  a  valuable  contribution  to  this  report. 

5.3.1  Aggregating  qualitatively  different  information 

The  literature  is  limited  with  respect  to  research  on  the  aggregation  of  different  types  of 
information  (e.g.,  visual  cues,  information  from  sensors,  etc).  However,  two  principles  were 
identified  which  offer  insight  into  how  integration  might  take  place,  including  linear  models  and 
the  majority  rule. 

5.3. 1. 1  Linear  models  and  multiple-cue  judgment 

Linear  modelling  describes  decision  making  as  an  additive  approach  where  cues  (information)  are 
assigned  weights  and  are  then  added  or  subtracted  based  on  their  directional  relationships  to  the 
judgment.  Qualitative  cues  are  thought  to  be  represented  in  binary  format  (0  =  gun  absent,  1  =  gun 
present)  so  that  the  linear  equation  can  be  calculated.  Linear  models  may  be  compensatory  or  non¬ 
compensatory.  Compensatory  models,  in  which  a  high  value  on  one  cue  can  compensate  for  a  low 
value  on  another,  require  that  every  cue  be  assessed,  weighted  and  combined  to  form  a  judgment, 
whereas  non-compensatory  models  do  not  require  that  every  cue  be  used  or  integrated  (Rothrock  & 
Kirlik,  2004).  In  a  compensatory  model,  no  single  cue  can  determine  the  outcome  because  a  subset 
of  cues  can  compensate  for  the  value  of  the  most  highly  weighted  cue.  Non-compensatory  models 
are  defined  by  the  absence  of  any  sub-set  of  cues  that  can  out-weigh  or  compensate  for  the  highest 
weighted  cue.  Compensatory  linear  models  are  part  of  analytic  decision  theory,  whereas  non¬ 
compensatory  models  are  more  associated  with  BDT. 

The  Lens  Model,  discussed  in  Section  4,  uses  compensatory  linear  modelling  to  examine  belief  and 
hypothesis  formation.  It  is  a  method  used  for  comparing  subjective  weightings  of  cues  to  the 
objective  (true  or  expert  determined)  weightings.  This  model  uses  a  proper  linear  model  to  show 
how  the  world  (criterion)  is  really  represented  by  cues  and  an  improper  linear  model  to  show  how 
people  actually  integrate  information.  The  discrepancy  between  these  models  is  used  to  determine 
how  people  make  errors  in  their  judgments  (Horrey  &  Wickens,  2001).  A  proper  linear  model  is  an 
analytic  method  of  describing  the  integration  of  information  such  that  the  weighing  of  information 
(predictor  variables)  forms  an  optimal  relationship  between  the  prediction  (judgment)  and  criterion 
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(standard  or  true  judgment).  An  improper  linear  model  is  non-optimal  because  people  rely  on 
intuitive  methods  to  weigh  cues  or  may  randomly  generate  weightings,  or  assign  all  cues  equal 
weightings  (Dawes,  1982).  Interestingly,  these  often  crude  improper  linear  models  have  been 
successful  at  predicting  close  to  optimal  judgments  (Dawes,  1982). 

Linear  models  are  used  in  a  formal  research  area  called  multiple-cue  judgment.  This  research, 
introduced  in  the  1950s,  evaluates  how  people  use  multiple  binary  or  continuous  cues  to  form  a 
judgment  (Juslin,  Olsson,  &  Olsson,  in  press).  It  is  typically  rooted  in  analytic  theory,  using 
statistical  methods  to  create  optimal,  compensatory  linear-additive  models.  A  classic  multiple-cue 
judgment  task  requires  participants  to  determine  whether  an  insect  is  toxic  based  on  learned  rules 
about  the  insect’s  leg  length  and  colouring.  Often,  the  cues  are  binary  (e.g.,  long  or  short  legs, 
spots  or  no  spots).  Two  models,  cue-abstraction  and  exemplar  memory,  have  been  developed  to 
explain  how  information  aggregation  is  performed  to  make  a  judgment. 

The  cue-abstraction  model,  an  example  of  analytic  decision  theory,  is  used  to  explain  how  people 
learn  the  rules  governing  the  relationship  between  the  cues  and  criterion.  During  training,  it  is 
assumed  that  people  mentally  integrate  these  rules,  which  are  later  probed  when  exposed  to  certain 
cues  in  the  environment.  Therefore,  the  cues  are  connected  to  the  criterion  in  memory,  providing  the 
direction  of  the  relationship  and  the  cue  weight  (Juslin  et  al.,  in  press).  The  final  judgment  (or 
aggregate),  if  binary  (e.g.,  toxic,  not  toxic),  is  based  on  the  sum  of  the  binary  cues  multiplied  by  their 
weight.  If  the  final  judgment  is  to  be  continuous  (e.g.,  the  bug  is  toxic  if  the  poison  concentration  is 
greater  than  55  ppm),  then  the  running  total  of  the  criterion  estimate  is  updated  with  each  additional 
cue.  Therefore,  this  model  suggests  that  people  integrate  information  based  on  learned  rules  that 
relate  cues  (e.g.,  gun/no  gun,  helmet/no  helmet)  to  the  criterion  (e.g.,  friend  or  enemy). 

The  exemplar  memory  model,  found  in  categorization  literature,  is  not  concerned  with  assigning 
weights  to  cues,  but  instead  relies  on  feature  matching  between  an  observed  object  and  those  found 
in  memory.  This  model  suggests  that  people  retrieve  specific  memory  traces  of  objects  from 
different  categories  (called  exemplars)  to  assist  them  in  making  judgments.  For  example,  people 
may  be  trained  by  memorizing  different  images  of  insects  that  are  categorized  as  toxic  or  not  toxic. 
When  they  next  see  an  insect,  its  features  will  be  used  to  retrieve  memories  of  insects  with  the 
same  features.  The  probability  that  the  insect  will  be  categorized  as  toxic  equals  the  “ratio  between 
the  summed  similarity  of  the  judgment  probe  to  the  toxic  exemplars,  and  the  summed  similarity  of 
all  exemplars”  (Juslin  et  al.,  in  press,  p.  3).  This  model  is  very  similar  to  the  pattern  matching 
strategy  described  in  Recognition-Primed  Decision  Making.  In  both  models,  experience  is  required 
to  possess  an  exemplar  of  a  similar  object  or  a  schema,  script  or  prototype  for  a  similar  situation. 
RPD  offers  additional  methods  such  as  mental  simulation  to  fit  the  observed  entity  or  situation  with 
a  previously  experienced  scenario. 

Both  the  cue-abstraction  and  exemplar  memory  models  may  be  useful  for  CID.  In  certain 
circumstances,  exemplar-based  training  may  be  effective.  For  example,  it  is  possible  that  people 
would  benefit  from  training  on  how  to  categorize  weapons  based  on  whether  they  are  used  by  friends 
or  enemies.  Karsh  et  al.  (1995)  trained  participants  to  recognize  friends  and  enemies  by  using  cue- 
cards  showing  various  angles  of  different  kinds  of  military  vehicles.  Participants  were  able  to 
effectively  discriminate  the  vehicles  in  terms  of  identity  in  a  computerized  task  after  20  minutes  of 
training.  Rule-based  learning,  as  described  with  the  cue-abstraction  model  may  also  be  applicable  in 
some  situations  if  rules  can  be  created  to  describe  the  relationship  between  cues  and  identity. 
However,  the  continuously  changing  battlefield  experienced  by  today’s  military  limits  the  ability  to 
create  such  rules  that  may  have  been  possible  in  the  past  (e.g.,  all  enemies  wear  a  specific  uniform). 
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5. 3. 1.2  Majority  Rule 

Voting  is  a  model  of  information  aggregation  useful  for  binary  decision  making.  As  already 
discussed,  it  is  useful  for  acquiring  private  information  held  by  individuals.  Condorcet’s  Jury 
theorem  states  that  choosing  to  use  the  majority  vote  as  the  final  decision  is  the  most  efficient 
method  of  aggregating  information  from  across  a  group  of  people  as  the  number  of  voters  increases 
towards  infinity  (Piketty,  1999;  Wit,  1998).  This  theorem  has  several  limiting  assumptions, 
including:  the  decision  must  be  a  categorical  binary  judgment  (e.g.,  friend  or  foe),  the  individuals 
or  sources  of  information  are  independent,  each  person  has  an  accuracy  level  of  greater  than  50 
percent,  and  they  are  voting  sincerely,  therefore  only  voting  for  a  judgment  if  they  have  private 
information  specific  to  that  option  (Piketty,  1999;  Feddersen  &  Pesendorfer,  1999).  Only  with 
large  enough  electorates  or  samples  does  this  theory  prevail  when  these  assumptions  are  violated 
(Piketty,  1999).  Although  voting  may  be  considered  the  formation  of  a  collective  decision, 
therefore  being  related  to  group  decision  making,  it  is  in  fact  directly  related  to  individual  decision 
making.  It  is  often  the  case  where  a  group  of  people  will  vote  or  provide  their  opinion  resulting  in 
a  collective  decision,  which  is  then  considered  by  the  individual  decision  maker  as  one  piece  of 
information  amongst  others.  For  example,  in  everyday  life,  a  decision  maker  might  poll  several 
friends  to  determine  whether  a  specific  restaurant  is  a  good  place  to  eat.  The  majority  vote  may 
result  in  a  “yes.”  However,  the  decision  maker  may  use  this  as  one  piece  of  information  that  is 
weighted  along  with  formal  restaurant  reviews.  It  is  not  known  how  often  soldiers  have  an 
opportunity  or  a  need  to  take  a  “vote”  regarding  the  identification  of  someone  that  they  encounter 
in  a  battlefield  scenario.  It  is  possible  that  the  fast  pace  of  such  a  situation  might  prevent  this 
method  of  aggregation.  However,  fireground  commanders,  who  deal  with  similar  high-pressured 
situations  have  reported  requesting  information  from  as  many  people  as  possible  in  order  to  gain  a 
better  awareness  of  the  situation  (Lipshitz  et  al.,  in  press).  Whether  this  information  was  a  vote  on 
the  proper  action  to  take  remains  to  be  determined. 

5.3.2  Aggregating  quantitative  information 

The  majority  of  information  aggregation  models  identified  are  specific  to  quantitative  information. 
Although  CID  decision  makers  must  combine  qualitatively  different  information,  Dean  et  al. 

(2005)  suggests  that  each  information  source  is  associated  with  a  probability  or  quantity  that  needs 
to  be  aggregated  to  determine  the  confidence  one  should  have  in  the  proposed  identity. 

Three  main  categories  of  models  became  apparent  when  reviewing  the  literature.  We  categorized 
them  as  follows: 


Information  Aggregation  Category 

Examples 

Reasoning  Rules 

Bayes’  Rule 

Measures  of  Centrality 

Averaging  (mean) 

Weighted  average 

Median 

Choosing  Strategies 

Take-the-best  heuristic 
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5.3.2. 1  Reasoning  Rules  -  Bayes  Rule  (Theorem) 

Bayes’  rule  uses  conditional  probability  theory  to  determine  the  optimal  way  to  update  prior  beliefs 
in  the  presence  of  new  information,  therefore  updating  prior  probabilities  (Budescu  &  Yu,  2006). 

It  assumes  that  the  probability  of  an  event  (i.e.  the  degree  of  belief)  is  dependent  upon  the  presence 
of  other  events  and  the  causal  links  between  them  (Van  Gosliga  &  Jansen,  2003). 

Bayes’  theorem  is  calculated  as  follows: 

p(h  |  e)  x  p(e)  =  p(e  \  h )  x  p(h) 

•  p(h  |  e)  is  the  posterior  probability,  which  is  the  probability  of  hypothesis  or  belief  “k”  after 

considering  the  effect  of  evidence  (i.e.  information,  cue)  “e”. 

•  p(e)  is  the  probability  of  observing  “e”. 

•  p(e  |  h)  is  the  likelihood,  which  provides  the  probability  of  circumstantial  evidence  “e” 
assuming  “h”  is  true. 

•  p(h)  is  the  prior  probability  of  “h”  without  observing  evidence  “e”. 

In  the  presence  of  additional  evidence  (e2),  the  calculation  becomes: 

p(h  |  el )  x  p(e  1 1  h,el ) 

Pie  1 

Bayes’  theorem  is  designed  to  deal  with  binary  information  (e.g.,  presence  or  absence  of  friendly 
insignia)  and  binary  hypotheses  (e.g.,  identity  is  friend  or  enemy).  This  rule  is  highly  utilized  to 
explain  and  evaluate  Economic  theories,  and  is  used  in  Computer  Science  as  a  data  fusion 
algorithm.  Bayes’  rule  is  a  normative  approach,  which  implies  that  it  describes  what  people  should 
do,  not  necessarily  what  they  do. 

There  is  conflict  in  the  literature  regarding  whether  people  are  capable  of  using  this  rule.  A  typical 
finding,  according  to  Budescu  &  Yu  (2006)  is  that  people  rely  too  heavily  on  prior  information  and 
do  not  adequately  update  their  assessment  with  respect  to  new  information,  therefore  failing  to  use 
Bayes’  rule.  However,  Sedlmeier  and  Gigerenzer  (2001),  on  the  other  hand,  have  been  able  to 
teach  people  to  use  Bayesian  reasoning  within  two  hours  of  training,  provided  that  they  use 
frequencies  instead  of  probabilities  during  the  task.  They  suggest  that  the  difficulty  associated  with 
Bayes’  Theorem  may  be  due  to  human  limitations  related  to  manipulating  probabilities. 

Budescu  and  Yu  (2006)  have  explored  whether  people  are  more  likely  to  use  a  Bayesian  approach 
or  an  averaging  model  to  aggregate  information.  They  used  a  medical  scenario  where  participants 
had  to  distinguish  between  two  diseases  based  on  a  subset  of  six  diagnostic  tests.  Participants 
made  a  diagnosis  and  provided  a  confidence  level  associated  with  their  judgment.  Three 
participants  were  selected  to  be  advisors  and  their  information  was  given  to  other  participants  who 
used  their  predictions  to  form  their  own  judgments  about  the  diagnosis.  The  advisors  had  different 
levels  of  independence  depending  on  whether  they  used  a  partial  overlap  of  the  same  diagnostic 
tests  or  used  completely  different  tests  when  making  their  judgments.  Budescu  and  Yu  (2006) 
tested  whether  participants  made  use  of  a  naive  Bayesian  approach  whereby  they  assumed  a 
conditional  independence  between  advisors  (information  sources),  therefore  treating  advisors’ 
predictions  as  perfectly  reliable  and  calibrated,  or  whether  they  used  an  averaging  method.  Briefly, 
averaging  weighs  each  piece  of  information  as  the  inverse  of  the  total  number  of  pieces  of 
information  therefore  reducing  the  effect  of  each  advisor.  A  naive  Bayesian  approach,  on  the  other 
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hand,  causes  the  contribution  of  each  new  piece  of  information  to  be  added  to  the  others  without  an 
adjustment  for  the  total  amount  of  information.  It  therefore  predicts  an  extremeness  effect  in 
contrast  to  the  averaging  compromise  effect  (Budescu  &  Yu,  2006).  Results  showed  that  averaging 
models  (especially  a  weighted  average)  were  superior  to  the  naive  Bayesian  model.  In  fact,  even 
when  participants  knew  that  advisors  were  independent  (i.e.  used  different  diagnostic  tests),  which 
is  a  condition  of  Bayes’  rule,  the  naive  Bayesian  model  fit  less  well  than  averaging  models. 
Participants  were  not  optimally  sensitive  to  inter-cue  correlations  or  the  level  of  inter-judge 
information  overlap.  Results  did,  however,  indicate  that  people  are  able  to  use  a  naive  Bayesian 
approach  in  certain  circumstances,  particularly  when  the  sources  of  information  are  in  close 
agreement.  When  advisors  provide  similar  diagnoses,  especially  when  the  judgments  are  more 
extreme,  people  are  more  likely  to  have  a  higher  level  of  confidence  in  the  judgment,  and  therefore 
are  comfortable  putting  full  weight  on  each  estimate.  However,  when  the  advisors  provided 
different  diagnoses,  people  had  a  lower  level  of  confidence  and  instead  used  an  averaging  method. 
Therefore,  according  to  Budescu  and  Yu  (2006),  the  method  of  aggregation  implemented  depends 
upon  the  perception  of  the  disagreement  between  different  sources  of  information. 

5. 3. 2. 2  Measures  of  Centrality 

5.3.2.2.1  Averaging 

The  averaging  model  is  a  conservative  approach  for  aggregating  quantitative  estimates.  It  involves 
adding  together  the  estimates  and  dividing  them  by  the  total  number  of  information  sources.  This 
is  also  termed  a  compromise  principle  because  each  piece  of  information  receives  less  weighting  as 
the  number  of  sources  increases.  The  averaging  method  is  a  form  of  linear  model  using  equal 
weights  (Budescu  &  Yu,  2006).  Budescu  and  Yu  (2006)  have  shown  that  this  method  is  used 
particularly  when  the  different  sources  of  information  are  perceived  to  have  a  large  discrepancy, 
causing  a  low  level  of  confidence  in  the  sources.  By  using  an  averaging  method,  the  discrepancy 
can  be  minimized. 

Despite  the  simplicity  of  averaging,  it  may  not  be  as  intuitive  as  it  sounds.  Larrick  and  Soil  (2006) 
performed  four  experiments  to  determine  that  people  do  not  intuitively  believe  that  averaging 
individual  estimates  from  various  sources  will  result  in  a  more  accurate  estimate.  Instead,  the 
majority  of  people  tested  held  a  false  theory  that  averaging  the  estimates  would  result  in 
performance  only  equal  to  that  of  the  average  individual.  Statistically  this  is  not  true. 

“When  combining  uncertain  quantity  estimates,  the  discrepancy  between  the 
average  estimate  and  the  truth  can  be  no  greater  than  the  average  discrepancy  of 
the  component  estimates.  ”  (Larrick  &  Soil,  2006,  p.  2). 

For  example,  if  in  reality  a  detected  entity  is  1000  meters  away  and  one  source  estimates  it  to  be 
1200  meters  away,  while  another  estimates  it  at  1010  meters,  then  the  average  estimate  is  1105 
meters.  This  average  estimate  is  105  meters  off  from  the  1000  meter  distance.  However,  the 
average  discrepancy  of  the  component  estimates  (1200-1000  =  200,  and  1010-1000  =  10)  is  105. 
Therefore,  the  average  estimate  is  always  equal  or  less  than  the  average  discrepancy.  Had  each  of 
the  sources  been  trusted  equally  and  only  the  most  discrepant  one  used,  then  the  discrepancy  could 
have  been  much  larger.  Therefore,  using  the  average  estimate  will  result  in  a  more  accurate 
estimate. 
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In  contrast  to  the  findings  of  Larrick  and  Soli  (2006),  Budescu,  Rantilla,  Yu  and  Karelitz  (2003) 
found  that  people  do  use  an  averaging  strategy.  In  their  study,  69%  of  participants  identified  the 
averaging  strategy  as  their  method  of  information  aggregation,  suggesting  an  introspective 
sensitivity  to  this  method. 

Interestingly,  one  factor  affecting  whether  people  are  likely  to  use  the  averaging  integration  model 
is  their  culture.  Individualistic  cultures,  such  as  the  United  States  and  Canada,  are  more  likely  to 
view  averaging  as  causing  a  loss  of  information  (i.e.  watering  down)  when  an  expert  is  included 
amongst  the  information  sources  than  are  collectivist  cultures  (Larrick  &  Soil,  2006). 

5.3.2.2.2  Weighted  Average 

Budescu,  Rantilla,  Yu  and  Karelitz  (2003)  found  that  when  people  aggregate  the  probabilistic 
opinions  from  advisors  who  use  asymmetric  numbers  of  cues  (e.g.,  test  results,  visual  symptoms, 
etc.,  when  forming  a  medical  diagnosis)  to  form  a  judgment,  they  alter  their  weighting  of  the 
information  based  on  both  the  number  of  cues  used  by  the  advisors,  and  the  known  accuracy  levels 
of  the  advisors.  Using  medical  and  business  scenarios,  Budescu  et  al.  (2003)  found  that  when  the 
decision  makers  (participants)  were  provided  with  information  from  3  advisors  who  had  varying 
degrees  of  asymmetry  (i.e.  the  advisors  used  different  numbers  of  cues  and  had  different  levels  of 
accuracy)  across  trials,  the  participants  were  more  confident  in  their  aggregate  when  the  advisors 
had  partially  redundant  information  as  well  as  when  the  information  was  distributed  across  advisors 
unequally.  Therefore,  some  level  of  redundancy  in  cues  used  by  the  advisors  was  necessary,  but 
the  advisors  should  not  use  all  of  the  same  cues  so  that  they  have  some  unique  information 
(Budescu  et  al.,  2003;  Yaniv,  2004). 

5.3.2.2.3  Median  Value 

When  people  have  no  previous  experience  with  specific  information  sources  they  are  likely  to  take 
the  median  estimate,  therefore  eliminating  outlying  opinions  to  accept  the  consensus  value.  Harries, 
Y aniv  and  Harvey  (2004)  performed  a  study  where  participants  imagined  they  were  in  a  foreign 
country  receiving  estimates  about  the  distance  and  age  of  a  nearby  city  from  four  other  people. 
These  estimates  were  said  to  be  based  on  individuals’  personal  knowledge  about  the  location.  One 
of  the  estimates  was  four  standard  deviations  from  the  mean  response.  The  participants  used  these 
estimates  to  make  a  decision  about  the  distance  and  age  of  the  city.  Harries  et  al.  (2004) 
determined  the  fit  of  the  estimates  and  found  that  people  eliminated  the  extreme  estimate  by 
intuitively  using  the  median  value.  The  ease  of  determining  the  median  was  suggested  as  a  primary 
reason  why  people  might  gravitate  towards  this  model,  in  addition  to  a  desire  to  produce  cognitive 
consonance  (Harries  et  al.,  2004). 

5. 3. 2. 3  Choosing  Strategy 

People  often  fail  to  use  information  aggregation  strategies  and  instead  choose  a  single  opinion, 
estimate  or  judgment  provided  by  an  advisor  or  information  source  (Soil  &  Larrick,  2005,  as  cited 
in  Larrick  &  Soil,  2006).  Participants  may  seek  to  use  the  estimate  of  what  they  perceive  to  be  the 
most  accurate  advisor  or  source  of  information,  or  the  advisor  that  used  the  most  cues  when 
forming  a  judgment  (Budescu  et  al.,  2003).  For  example,  a  soldier  may  judge  a  vehicle  as  friendly 
because  of  the  report  of  “friend”  from  an  IFF  sensor,  without  utilizing  judgments  from  other 
sources,  or  by  making  use  of  personal  information  based  on  experience  and  visual  cues.  In  other 
situations,  people  may  seek  to  use  only  their  own  information  instead  of  that  provided  by  others 
(Yaniv  and  Milyavsky,  in  press),  as  will  be  discussed  in  Section  5.4.3.  This  strategy  is  not 
restricted  to  quantitative  estimates,  but  can  also  be  applied  to  qualitatively  distinct  cues. 
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A  heuristic  similar  to  the  choosing  strategy  is  “Take-the-Best”  (TTB).  This  is  a  fast  and  frugal 
heuristic  that  makes  use  of  only  the  single  most  valid  cue  to  make  a  judgment  between  two  options 
(Gigerenzer,  2004;  Jones,  Juslin,  Olsson  &  Winman,  in  press).  The  validity  of  a  cue  is  determined 
by  the  ratio  of  the  number  of  correct  predictions  made  by  using  a  cue,  to  the  number  of  pairs  where 
the  values  of  a  cue  differ  between  objects  being  discriminated.  For  example,  if  a  person  needs  to 
judge  whether  a  person  is  a  friend  or  foe,  they  will  acquire  a  variety  of  cues.  When  using  the  TTB 
heuristic,  the  cue  previously  determined  to  have  the  highest  validity  will  be  examined  first  for  its 
value.  In  a  CID  context,  this  cue  might  be  the  presence  or  absence  of  a  helmet.  If  there  is  no 
helmet  present,  the  cue  will  not  provide  enough  information  to  adequately  discriminate  between 
friend  or  foe  (e.g.,  the  person  may  be  a  friend  by  not  wearing  the  full  uniform).  Therefore  another 
cue  will  be  examined.  If,  however,  there  is  a  helmet  present,  this  may  satisfactorily  identify  the 
person  as  a  friend,  and  the  search  will  end  (refer  to  Bryant,  2007  for  an  example  of  TTB  applied  to 
an  air  threat  decision  task).  Therefore,  the  TTB  heuristic  has  three  parts:  search  by  validity,  one- 
reason  stopping  rule,  and  one-reason  decision-making.  By  using  a  stopping  rule,  people  can  draw 
inferences  without  assessing  the  details  of  every  cue  (Gigerenzer,  2004). 

5.4  Weighing  information 

One  of  the  most  important  components  of  the  information  aggregation  process  is  assigning  weights 
to  the  information.  Wickens  et  al.  (1999)  describes  a  weighting  as  the  amount  of  attention  people 
give  to  a  specific  cue.  This  weighting  can  be  affected  by  many  factors  such  as  the  saliency  of  the 
cue,  the  association  of  the  cue  to  a  correct  choice  based  on  previous  experience,  the  trust  in  the  cue, 
and  more.  Here  we  will  review  general  weighting  strategies,  methods  used  to  handle  dissenting 
information,  self-versus-other  weightings,  and  methods  of  dealing  with  uncertainty  and  information 
overload. 

5.4.1  General  weighting  strategies 

Not  all  information  is  of  equal  value  and  therefore  should  not  be  treated  as  such  when  forming  a 
judgment.  Trust,  certainty,  relevance  and  importance  of  cues  all  affect  how  people  subjectively 
weigh  cues  and  sources  of  information  (Yaniv,  2004;  Wickens  at  el.,  1999;  Horrey  &  Wickens, 
2001). 

Trust  level  and  certainty  associated  with  a  cue  is  related  to  the  accuracy  of  the  information  source, 
the  expertise  of  the  advisor  in  the  case  of  opinion  aggregation,  the  confidence  reported  by  the 
source,  and  previous  performance  (Yaniv,  2004).  For  example,  when  people  repeatedly  experience 
accurate  opinions  from  a  particular  advisor  even  when  the  opinion  disagrees  with  other  advisors, 
they  will  be  more  likely  to  accept  this  dissenting  information,  despite  the  tendency  to  choose 
consensus  information  (Harries  et  al.,  2004).  In  the  case  of  CID,  there  are  factors  that  may  affect 
the  formation  of  trust  for  a  given  information  source,  such  as  the  reliability  of  the  sensors  (Wickens 
et  al,  1999).  IFF  sensors  can  be  subject  to  error  (e.g.,  signal  is  acquired  from  a  neighbouring 
vehicle,  not  the  detected  entity)  which  when  experienced,  will  effect  the  weighting  people  apply  to 
this  information  in  the  future.  Individual  differences  will  affect  trust  levels  because  not  everyone 
will  experience  the  same  errors  and  mistakes,  therefore  altering  how  they  each  weigh  different 
sources  of  information.  Within  Lens  Model  terminology,  this  difference  is  discussed  in  terms  of 
the  calibration  between  the  subjective  and  objective  weighting  of  the  cues  (Wickens  et  al.,  1999). 
Individuals  will  have  different  beliefs  about  the  value  of  each  piece  of  information,  which  will 
affect  how  they  allocate  their  attentional  resources  to  each  cue.  Wickens  et  al.  (1999)  discuss  two 
forms  of  calibration  to  describe  how  people  weigh  information:  Attention  (A)  Calibration  and  Trust 
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(T)  Calibration.  These  calibrations  refer  to  the  best  way  to  allocate  attention  and  trust  to  the  cues 
or  to  the  various  hypotheses  (alternatives).  Whenever  these  calibrations  are  violated,  there  is  a 
Type  A  (attention)  or  T  (trust)  bias.  A  Type  A  bias  occurs  when  people  allocate  their  attention 
between  sources  in  an  inappropriate  way  given  the  true  value  of  sources  or  the  cost  of  ignoring  a 
source  or  hypothesis  (importance).  The  Type  T  bias  occurs  when  people  have  an  overtrust  in  a 
source  and  therefore  respond  to  a  cue  that  is  unreliable,  or  when  they  have  an  undertrust  causing 
them  to  ignore  relevant  cues  due  to  a  previously  experienced  false  alarm  (Wickens  at  el.,  1999). 

Relevance  and  importance  of  the  cue  will  also  affect  the  weighting.  Horrey  and  Wickens  (2001) 
found  that  people  weighed  (and  therefore  processed)  relevant  information  more  heavily  than 
irrelevant  information,  as  revealed  during  a  memory  probe  test.  The  importance  of  a  cue  can  be 
affected  by  its  salience,  such  as  the  size  or  intensity  in  a  display  (Wickens  et  al.,  1999).  Automatized 
cueing  in  a  display  can  draw  attention  to  appropriate  cues  eliciting  a  greater  weight  (Horrey  & 
Wickens,  2001).  Importance  is  also  determined  through  experience.  As  discussed  by  the  members 
of  1-PPCLI,  who  had  returned  from  Afghanistan,  with  HSI®’s  T.  Lamoureux  (Feb.  9,  2007, 
Edmonton,  Alberta),  experience  with  the  daily  routine  of  villagers  assists  with  assessing  the  situation 
and  therefore  recognizing  when  a  danger  might  be  present.  Without  such  experience,  it  would  be 
difficult  to  complete  the  OODA  loop  prior  to  that  of  hostile  forces.  By  understanding  the  culture  and 
environment,  importance  is  assigned  to  the  presence  or  absence  of  certain  cues.  Yager  (1993)  has 
proposed  weighing  information  based  on  the  relevance  it  has  to  the  decision  at  hand  by  using  a 
penalty  approach.  To  do  this,  people  would  assign  a  penalty  value  to  every  piece  of  information 
based  on  what  it  would  cost  if  the  information  were  disregarded  therefore  resulting  in  a  decision  that 
conflicted  with  the  information.  Haspert  (2000)  has  suggested  incorporating  the  cost  of  committing 
fratricide  into  the  weighing  process  for  CID.  This  cost  will  vary  depending  on  the  intensity  of  the 
situation,  therefore  affecting  the  threshold  required  (certainty  level)  to  make  an  identification. 

Typical  weighting  strategies  suggested  in  the  CID  literature  are  probabilities  associated  with  the 
likelihood  that  a  piece  of  information  is  accurate  and  will  lead  to  the  correct  identification,  which 
when  aggregated  results  in  an  overall  confidence  in  the  judgment  (Dean  et  al.,  2004).  These 
probabilities  therefore  may  be  more  focused  on  the  trust  aspect  of  weightings.  However,  CID 
decision  makers  will  also  need  to  decide  which  information  to  use  in  the  aggregation,  which  is 
likely  to  be  related  to  relevance  and  importance.  Karsh  et  al.  (1995)  found  that  military  personnel 
performing  in  a  tank  identification  simulation  neglected  to  use  all  available  information  sources, 
despite  reasonable  accuracy  levels.  This  suggests  that  they  did  not  view  the  additional  sources  as 
important  in  contrast  to  other  cues. 

5.4.2  Weighing  Dissenting  Information 

In  many  decision-making  situations,  information  from  one  source  may  contrast  with  that  provided 
by  other  sources.  Decision  makers  must  determine  how  to  handle  this  conflicting  information. 

One  could  imagine  a  CID  scenario  where  the  ID  returned  from  a  sensor  during  a  vehicle 
interrogation  may  claim  “friend”  and  yet  all  other  indications  suggest  enemy.  This  could  occur  if  a 
friendly  force  was  nearby  an  enemy  and  therefore  its  transponder  was  returning  the  friendly  code. 
The  commanding  decision  maker  must  choose  whether  to  accept  this  contradictory  information, 
consider  it  with  more  or  less  weighting  when  aggregating  all  available  information  or  omit  it 
entirely.  Although  the  rules  of  engagement  are  generally  designed  to  assist  in  these  kinds  of 
situations,  it  is  important  to  understand  the  typical  cognitive  response  to  this  kind  of  information. 
According  to  Harries,  Yaniv  and  Harvey  (2004),  people  often  discount  outlier  opinions  because  of 
a  cognitive  drive  to  reach  consensus.  By  trimming  information  to  focus  on  the  majority  response 
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one  can  avoid  creating  dramatically  obscured  aggregated  estimates  due  to  an  information  source 
providing  strategically  exaggerated  or  erroneous  information.  However,  resolving  inconsistencies 
by  ignoring  them  may  be  a  risky  practice  that  can  eliminate  important  information  unavailable 
from  other  sources.  Harries  et  al.  (2004)  found  that  when  people  are  provided  with  quantitative 
estimates  (e.g.,  age  of  or  distance  to  a  city)  where  one  of  the  estimates  was  four  standard  deviations 
away  from  the  others,  people  chose  the  median  response,  therefore  disregarding  the  outlying 
estimate.  Harries  et  al.  (2004)  determined  that  the  only  way  to  overcome  this  default  response, 
therefore  causing  selection  of  the  outlying  information  as  the  estimate  of  choice,  required  that 
people  experience  repeated  feedback  (as  many  as  1 0  trials)  regarding  the  pairing  between  a 
particularly  named  source  and  an  accurate  outlier.  Therefore,  people  can  learn  to  utilize  dissenting 
information,  but  it  takes  time  and  experience. 

Military  research  has  also  suggested  that  people  dismiss  inconsistent  information  when  making  a 
decision.  Perrin,  Barnett,  Walrath,  and  Grossman  (2001)  studied  how  U.S.  Navy  personnel  identified 
unknown  aircraft.  Information  inconsistent  with  their  judgment  was  recalled  less  often  than 
consistent  information,  suggesting  that  the  inconsistent  information  was  weighted  and  processed  less. 
It  is  also  possible  that  the  inconsistent  information  was  reinterpreted  to  fit  with  a  known  pattern  such 
as  a  schema,  script  or  prototype  already  present  in  the  experienced  personnel’s  cognitive  repertoire. 
The  study  was  unable  to  discriminate  between  these  possibilities.  Another  example  involving 
inadequate  use  of  inconsistent  information  was  the  USS  Vincennes  shooting  of  Iran  Flight-655.  One 
factor  causing  the  error  was  the  IFF  reading  of  both  COMAIR  (commercial  airline)  and  MIL  AIR 
(military).  The  COMAIR  report  did  not  fit  with  the  other  information  (e.g.,  failure  to  respond  to 
warnings)  and  therefore  was  not  weighed  properly  (Craig,  Morales,  &  Oliver,  2004). 

Ultimately,  people  desire  to  explain  away  the  differences  between  sources  by  using  cues  about 
accuracy  and  credibility  (Yaniv,  2004).  This  is  observed  in  such  social  arenas  as  criminal  court 
where  a  lawyer  may  attempt  to  discredit  a  source  of  information  because  it  does  not  fit  with  the 
story  being  developed  to  achieve  a  judgment  of  innocence.  In  a  CID  scenario  such  as  the  search 
for  a  Taliban  leader  in  an  Afghan  village,  a  sleeping  person  found  holding  a  gun  will  cause  conflict 
when  forming  a  judgment  about  the  identification  of  the  person,  because  although  they  may  be 
hostile  when  awake,  this  cannot  be  determined  with  confidence  in  the  presence  of  these  two  cues. 

CID  decision  makers  will  have  to  rapidly  decide  whether  to  ignore  the  sleeping  person  or  to 
consider  the  person  hostile  despite  the  possibility  that  the  gun  was  held  during  sleep  solely  for 
protection  against  such  enemies  as  the  Taliban. 

5.4.3  Egocentric  Weighing  of  Information 

If  we  expand  upon  the  above  sleeping-with-a-gun  scenario  so  that  this  person  is  encountered  by  a 
group  of  three  soldiers  quietly  manoeuvring  through  an  area  during  a  late-night  raid,  we  can 
present  an  example  of  decision  making  influenced  by  an  egocentric  mindset.  For  example,  it  is 
possible  that  two  of  the  soldiers  will  identify  the  person  as  neutral  based  on  their  understanding  of 
the  danger  associated  with  this  village  and  the  need  for  self-defence.  Although  they  may  signal  this 
opinion  to  the  third  soldier,  he  may  still  choose  to  identify  the  person  as  hostile  because  of  his 
interpretation  of  the  scenario.  Instead  of  integrating  the  opinions  of  the  other  soldiers,  the  third 
decision  maker  may  be  subject  to  an  overconfidence  bias  causing  him  to  assess  his  opinion  as 
being  more  accurate  than  that  of  the  others. 

The  overconfidence  bias  is  thought  to  occur  because  people  can  access  the  information  that  led  to 
their  judgment,  whereas  the  cues  used  by  others  are  not  so  easily  determined.  Yaniv  (2004)  has 
reported  that  people  have  been  found  to  place  70%  weighting  on  their  own  opinion  and  30%  on  the 
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opinions  of  others.  Additionally,  Y aniv  and  Milyavsky  (in  press)  have  shown  that  people  give 
more  weight  to  opinions  that  are  closer  to  their  own  than  those  that  are  distant.  The  literature  does 
not  specify  how  distant  an  opinion  must  be  to  be  ignored,  but  does  suggest  that  the  opinions  or 
estimates  that  differ  the  most  from  one’s  own  opinion  may  be  subject  to  this  exclusion.  Therefore, 
the  greater  the  distance  between  advice  and  one’s  own  opinion,  the  less  likely  it  will  be  considered. 
This  is  referred  to  as  egocentric  trimming.  Yaniv  and  Milyavsky  (in  press)  explain  that  from  an 
objective  or  external  perspective,  opinions  from  others  and  self  appear  equal.  However,  from  an 
internal  or  subjective  point  of  view,  opinions  are  not  perceived  as  equal  because  the  decision  maker 
has  more  evidence  available  to  mind  regarding  his  opinion  than  he  does  of  the  advisor’s  view.  This 
effect  of  weighing  own  information  greater  than  others,  particularly  when  others  have  dissenting 
views,  is  also  related  to  the  confirmatory  bias  related  to  belief  revision  (Keren  &  Teigan,  2004; 
Westen  &  Weinberger,  2004).  When  people  have  a  strong  opinion  or  belief  about  the  state  of  the 
world,  they  are  more  likely  to  seek  evidence  or  cues  that  confirm  this  belief.  Evidence  in  contrast 
to  their  opinion  is  perceived  as  an  outlier  that  should  be  ignored. 

Yet  another  heuristic  associated  with  the  egocentric  frame  of  mind,  is  the  Availability  Heuristic  (Ross 
&  Sicoly,  1982).  Tversky  and  Kahneman  (1982b)  describe  the  availability  heuristic  as  causing 
people  to  deem  something  more  likely  to  happen  depending  on  whether  it  is  easy  to  bring  a  similar 
instance  to  mind.  This  heuristic  is  based  on  the  implicit  assumption  that  the  availability  of  examples, 
as  determined  by  its  rate  of  occurrence,  reflect  their  true  likelihood.  People  may  use  this  heuristic, 
especially  in  time-pressured  situations,  to  weigh  cues  differently  depending  on  how  they  relate  to 
instances  they  more  easily  recall  (due  to  active  or  passive  experience).  If  the  soldier  who  identifies 
the  sleeping  person  as  an  enemy  could  draw  upon  instances  where  people  with  hostile  intent  have 
faked  sleeping  before  attacking  then  this  would  cause  him  to  weigh  the  sleep  cue  more  lightly  than 
others  would,  and  may  lead  to  favouring  of  his  own  hostile  interpretation.  Another  possible  use  of  the 
availability  heuristic  is  related  to  choosing  to  use  cues  most  easily  brought  to  mind.  For  example,  the 
belief  that  people  wearing  helmets  are  friendly  because  of  repeated  observation  of  this  “rule”,  may 
cause  this  cue  to  be  weighted  more  heavily  when  observed,  even  if  there  are  other,  less  common  cues 
suggesting  hostile  intentions. 

Noth  and  Weber  (2003)  have  studied  the  effect  of  overconfidence  on  use  of  personal  versus  others’ 
information.  When  people  make  decisions  one  after  the  other,  therefore  having  an  opportunity  to  use 
their  private  information  and  that  of  the  people  who  have  made  judgments  before  them,  they  often  fail 
to  aggregate  the  information  in  a  rational  way.  Instead,  they  tend  to  be  overconfident  and  put  more 
weight  on  their  own  information,  even  when  they  know  that  their  information  is  weak  in  quality.  In 
their  study.  Noth  and  Weber  (2003)  had  participants  perform  a  computerized  experiment  where  they 
were  informed  of  their  private  information  (A  or  B),  the  strength  of  the  information  (strong  or  weak), 
and  provided  with  the  information  (A  or  B)  of  the  preceding  decision  makers  (if  there  were  any). 
Participants  chose  A  or  B  based  on  this  information.  They  performed  approximately  80  trials 
receiving  feedback  in  different  currency  units  depending  on  whether  their  choice  was  correct  or  not. 
Noth  and  Weber  (2003)  tested  2  methods  of  aggregation,  Bayesian  Updating  (BU),  whereby  people 
would  follow  Bayes’  rule  when  making  their  choice  or  Private  Information  (PI),  whereby  participants 
would  always  use  their  own  information  regardless  of  its  strength.  If  people  behaved  rationally,  they 
would  use  Baye’s  rule  in  conjunction  with  the  knowledge  of  their  position  in  the  sequence  of 
information  presentation.  See  Table  3  for  a  description  of  how  people  should  respond  based  on 
Bayes’  rule  compared  to  what  was  actually  observed  in  this  experiment.  The  results  revealed  use  of  a 
heuristic  based  on  overconfidence  in  private  information. 
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By  asking  participants  how  they  had  performed  the  task,  the  heuristic  was  named:  Modified  Counting 
Heuristic  (MCH).  MCH  works  as  follows  (Noth  &  Weber,  2003): 

•  Predict  according  to  own  strong  signal  if  in  position  1,  2,  3,  or  4  of  the  decision  maker 
sequence. 

•  Use  own  strong  signal  at  position  5  and  6  if  more  than  one  deviation  is  observed  across 
preceding  decision  makers,  otherwise,  go  with  the  majority  response. 

•  Predict  according  to  own  weak  signal  if  at  position  1  or  2  and  use  own  signal  only  if  no 
majority  exists  and  the  last  two  subjects  have  not  predicted  the  same  state.  Otherwise, 
go  with  the  majority  response. 

The  MCH  is  consistent  with  the  idea  of  overconfidence,  which  is  influenced  by  a  need  to 
overcompensate  for  the  expected  error  within  public  information. 


Table  3.  Expected  and  observed  judgments  in  a  sequential  information  presentation 

scenario. 


Position  in  the 
sequence  of 
decision  makers 

Type  of 

private 

information 

Preceding 

Information 

Response  According  to 

Bayes’  Rule 

Actual  Response 

1 

Weak 

None 

Choose  private  information 

14.2%  failed  to  choose  private 
information 

1 

Strong 

None 

Choose  private  information 

3%  failed  to  choose  private 
information 

2 

Weak 

Opposite  choice  of 
private  information 

Choose  choice  of  preceding 
decision  maker 

49.3%  chose  based  on  private 
information  (overconfidence) 

2 

Strong 

Opposite  choice  of 
private  information 

Choose  private  information 

97%  chose  based  on  private 
information 

3 

Weak 

Opposite  choice  of 
private  information 

Choose  choice  of  immediate 
predecessor  (particularly  if 
both  predecessors  make 
same  choice) 

73.7%  chose  based  on  private 
information  (overconfidence) 

3 

Strong 

Opposite  choice  of 
private  information 

Choose  private  information 

14.7%  failed  to  choose  their 
own  information  (irrational 
herding/conforming) 

One  CID  study  was  identified  that  provided  evidence  of  a  potential  overconfidence  bias.  Karsh  et 
al.  (1995)  found  that  CID  decision  maker’s  in  a  tank  identification  simulation  relied  more  on  their 
visual  information  than  on  that  provided  by  a  decision  support  system.  This  study  could  not 
confirm  the  presence  of  an  overconfidence  bias  because  the  decision  support  system  was  only  used 
in  14.5%  of  the  trials,  suggesting  that  it  might  have  been  available  too  infrequently  for  it  to  be 
considered  an  important  cue. 

5.4.4  Applying  Weights  in  the  Presence  of  Information  Overload 

Although  there  are  many  benefits  associated  with  using  multiple  pieces  of  information  when 
forming  a  judgment,  too  much  information  can  cause  difficulties.  A  large  number  of  cues  can  put 
demands  on  limited  resources  such  as  selective  and  divided  attention,  and  working  memory 
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(Wickens  &  Carswell,  1995  as  cited  in  Wickens,  Pringle  &  Merlo,  1999).  Although  there  is  no 
clear  explanation  as  to  what  qualifies  as  ‘too  much’  information,  the  classic  limits  of  working 
memory  (7  +1-2  units)  may  be  one  limiting  factor.  Time  will  also  be  a  factor,  because  most  cues 
will  require  attention  in  a  serial  manner,  therefore  limiting  how  much  information  can  be 
processed.  When  the  task  load  is  too  high,  people  may  either  fail  to  aggregate  all  of  the 
information,  or  they  may  provide  arbitrary  weights  to  the  information  to  avoid  evaluating  the  cues 
(Perrin  et  al.,  2001;  Horrey  &  Wickens,  2001).  For  example,  Perrin  et  al.  (2001)  increased  the 
number  of  aircraft,  display  clutter  and  number  of  communications  that  needed  to  be  monitored  by 
U.S.  Navy  personnel  participating  in  a  threat  assessment  task  and  found  that  participants  decreased 
the  amount  of  information  that  they  processed  in  the  high  demand  tasks.  This  was  revealed  by 
their  decrease  in  recalled  information  during  a  memory  probe  test  (Perrin  et  al.,  2001). 

When  so  much  information  is  present  that  people  are  unable  to  properly  weigh  it,  or  the  different 
sources  have  inconsistent  reliabilities,  people  may  utilize  the  “as  if’  heuristic.  This  means  that 
people  will  weigh  all  cues  “as  if’  they  were  the  same  (Wickens  et  al.,  1999;  Florrey  &  Wickens, 
2001).  Use  of  this  heuristic  can  cause  excess  weight  to  be  placed  on  cues  of  low  value  and  less  on 
those  of  high  value.  Wickens  et  al.  (1999)  suggest  avoiding  this  problem  by  drawing  people’s 
attention  towards  important  cues  by  presenting  them  in  a  salient  format  in  a  visual  display. 
Unfortunately,  this  method  will  not  always  be  possible,  especially  in  face-to-face  combat 
identification  scenarios. 

An  additional  way  people  cope  with  high  demand  is  through  pattern  matching  as  described  in  the 
RPD  model  (Florrey  &  Wickens,  2001).  People  with  domain-specific  experience  use  pattern 
matching  to  associate  recognized  cues  with  similar  situations  found  in  long-term  memory  (O’Flare 
&  Wiggins,  2004;  Klein,  1997).  These  memories  are  used  to  match  actions  to  the  situation. 
Gonzalez  (2001)  has  suggested  that  pattern  matching  of  RPD  is  basically  the  same  as  BDT’s 
Representativeness  Fleuristic. 

According  to  the  representativeness  heuristic,  we  judge  the  likelihood  of  an  uncertain  event 
according  the  degree  to  which  the  event  is  similar  to  the  population  from  which  it  is  derived  or  the 
degree  to  which  it  reflects  the  salient  features  of  the  process  that  generates  the  event  (Kahneman  & 
Tversky,  1982).  For  example,  CID  would  qualify  as  one  of  these  judgments  when  determining  the 
degree  to  which  a  person  resembles  a  suicide  bomber.  Klein  and  MacGregor  (1988)  describe  the 
representativeness  heuristic  as  a  method  for  evaluating  the  degree  to  which  a  particular  entity  has 
features  similar  to  ones’  learned  prototype.  This  heuristic  seems  to  be  similar  to  the  exemplar 
memory  model  described  previously  when  discussing  multiple-cue  judgment.  Flowever,  a  bias 
caused  by  using  the  representativeness  heuristic  is  that  people  fail  to  consider  the  probability  of  A 
or  B  present  in  the  situation.  It  is  therefore  as  if  people  are  using  Bayes’  rule  without  taking  into 
consideration  the  base-rates  (Kahneman  &  Tversky,  1982).  For  example,  in  a  location  where  there 
is  a  very  low  proportion  of  “friends”,  the  probability  of  detecting  a  friend  will  be  low.  However,  if 
the  person  appears  to  match  the  soldier’s  mental  representation  of  a  friend,  even  if  this  information 
is  weak,  then  they  may  be  biased  to  make  a  friend  identification,  even  though  they  should  also 
consider  the  actual  likelihood  of  this  identity.  Therefore,  the  representativeness  heuristic  can  result 
in  an  improper  weighing  of  information  such  that  situational  factors,  like  base  rates,  are  ignored. 

An  additional  heuristic  that  may  influence  how  CID  decision  makers  weigh  information  is  the 
Adjustment  and  Anchoring  Heuristic.  This  heuristic  causes  people  to  anchor  their  estimate  to  the 
first  or  most  recent  estimate  presented  by  someone  else.  In  this  case,  people  are  not  behaving 
egocentrically,  but  instead  are  reliant  upon  others’  judgments,  even  if  they  are  incorrect  (Tversky  & 
Kahneman,  1982a). 


Human. systems 


Combat  Identification:  Literature  Review 


Page  35 


HUM  A  NS  YSTE  MS 


5.4.5  Dealing  with  uncertainty 

Situations  may  also  occur  when  the  decision  maker  will  have  too  little  information.  This  will  cause 
the  formation  of  an  insufficiently  certain  aggregate,  which  will  reduce  the  possibility  of  making  a 
good  judgment.  Overall,  there  are  three  main  forms  of  uncertainty  that  may  affect  decision 
making,  including  (Lipshitz  et  al.,  2001a): 

1)  inadequate  understanding  of  the  situation  because  of  insufficiently  coherent  SA; 

2)  incomplete,  ambiguous  or  unreliable  information  (lack  of  information);  and 

3)  alternative  courses  of  action  are  insufficiently  differentiated  by  the  aggregated  information 
such  that  a  choice  cannot  be  easily  made. 

Lipshitz  et  al.  (2001a)  describes  two  tactics  used  in  NDM  to  improve  decision  making  in  the  midst 
of  uncertainty:  the  RAWFS  heuristic  and  Recognition/Meta-Cognition  (R/M)  model. 

The  Reducing,  Assumption,  Weighing,  Forestalling  and  Suppressing  (RAWFS)  heuristic  is  used  to 
deal  with  uncertainty  caused  by  either  insufficient  situation  awareness,  a  lack  of  information,  or 
conflicting  alternatives  (i.e.  COAs)  by:  reducing  uncertainty,  using  assumption-based  reasoning, 
weighing  the  pros  and  cons,  forestalling  and  suppressing  the  uncertainty.  Lipshitz  and  Strauss 
(1997)  as  cited  in  Lipshitz,  Sender,  Omodei,  McClellan,  and  Wearing  (in  press)  describe  these 
coping  strategies  as  follows: 

•  Reduction.  Reduce  uncertainty  by  searching  for  more  information  to  add  to  the 
aggregate.  This  method  is  consistent  with  the  reiteration  of  the  cycle  of  acquiring  and 
aggregating  information  in  the  1NCIDER  model  developed  by  Dean  et  al.  (2005)  when 
the  identification  has  not  reached  a  desired  level  of  confidence. 

•  Assumption-based  reasoning.  Use  previous  knowledge  and  imagination  to  make  sense 
of  the  available  information. 

•  Weighing  pros  and  cons.  Make  an  approximation  of  the  subjective  expected  utility  of 
each  COA  alternative. 

•  Forestalling.  Prepare  a  COA  to  counter  potential  negative  unforeseen  events. 

•  Suppressing  uncertainty.  Ignore  the  uncertainty  and  take  a  risk  by  proceeding  with  the 
COA. 

The  reduction  and  assumption-based  reasoning  strategies  most  directly  affect  information 
aggregation,  while  the  other  strategies  focus  on  choosing  amongst  alternatives.  When  the  situation 
does  not  make  sense,  people  may  seek  more  information,  therefore  reinitiating  the  information 
acquisition  phase  and  then  re-aggregating  the  larger  volume  of  information.  However,  when  there 
is  no  extra  information  available,  people  may  pull  on  previous  knowledge  to  imagine  what  the 
situation  might  mean,  therefore  combining  the  present  aggregate  with  previous  knowledge.  This 
may  involve  shifting  the  weights  associated  with  each  piece  of  information  so  that  an  acceptable 
decision  can  be  made. 

Lipshitz  et  al.  (in  press)  used  a  cued-recall  methodology  based  on  incidents  captured  by  fire¬ 
fighting  unit  commanders’  head-mounted  video  cameras  to  study  the  RAWFS  heuristic.  The  study 
required  that  the  commanders  recall  and  describe  what  was  going  on  in  their  minds  during  the 
recorded  incidents.  These  statements  were  coded  to  determine  the  kinds  of  uncertainty  the 
commanders  experienced  and  how  they  coped  with  it.  Inadequate  situational  awareness  and 
uncertainty  about  the  required  COA  and  the  cause  of  the  incident  were  the  main  sources  of 
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uncertainty.  Results  indicated  that  the  commanders  used  Reduction,  Assumption-based  reasoning 
and  Forestalling  to  cope  with  the  uncertainty.  Reduction  was  performed  by  delaying  action  (i.e. 
waiting  for  more  information  passively),  relying  on  standard  operating  procedures  (e.g.,  such  as 
ROE  for  CID  decision  makers),  prioritizing  (i.e.  attending  to  the  most  necessary  action  known), 
and  actively  searching  for  cues  through  all  senses  (e.g.,  seeking  cues  from  crew  members,  others  on 
the  scene,  etc).  Commanders  had  to  monitor  the  situation  continuously,  updating  and 
disconfirming  expectations.  Assumption-based  reasoning  involved  proceeding  forward  with  a 
plan,  using  mental  rehearsal  to  imagine  possible  COAs,  mental  simulation  to  determine  how  they 
would  turn  out,  and  using  assumptions  when  forming  situational  awareness.  Forestalling  did  not 
involve  specific  tactics.  Assumption-based  reasoning  was  also  used  on  the  way  to  the  incident 
such  that  fire  fighting  unit  commanders  listened  to  the  dispatcher  in  order  to  construct  a  mental 
model  of  the  situation  so  that  actions  that  might  be  required  could  be  anticipated  ahead  of  time. 

Fire  fighter  unit  commanders  relied  heavily  on  standard  operating  practices  to  guide  their  actions. 
They  also  relied  on  other  people,  such  as  subordinate  fire-fighters,  bystanders,  the  dispatcher  and 
other  units  at  the  scene  to  be  sources  of  information.  Therefore,  despite  the  time -pressured 
situation,  information  was  still  gathered  across  as  many  sources  as  possible  when  performing 
recognition-primed  decision  making.  Lipshitz  et  al.  (in  press)  found  that  the  commanders  first  used 
reduction  to  reduce  uncertainty  followed  by  assumption-based  reasoning  strategies  and  then 
forestalling,  leaving  the  weighing  of  pros  and  cons  and  suppression  as  last  resort  methods. 

A  second  tactic  is  the  Recognition/Meta-Cognition  model,  which  suggests  that  people  use  a  Story, 
Test,  Evaluate  and  Plan  (STEP)  procedure  when  their  recognition  or  pattern  matching  fails  because 
of  uncertainty.  The  STEP  process  includes  four  parts:  construct  a  Story,  Test,  Evaluate  and  Plan 
(Lipshitz  et  al.,  2001a).  Constructing  a  story  involves  using  whatever  fragment  of  recognition  is 
available  through  pattern  matching  to  make  an  assessment  of  the  nature  of  the  situation,  including 
past,  present  and  future  events  consistent  with  this  understanding.  Testing  is  done  to  determine  the 
likelihood  of  the  constructed  story  given  the  implications  and  expectations  associated  with  it  and 
how  these  relate  to  the  information  present.  When  gaps  or  inconsistencies  are  observed  in  the 
story,  the  decision  maker  revises  it.  The  Evaluate  phase  is  described  as  a  “devil’s  advocate” 
approach  where  the  decision  maker  attempts  to  justify  and  explain  the  assumptions  associated  with 
the  story  as  they  fit  with  the  information  about  the  present  situation.  Finally,  the  Plan  phase  is 
similar  to  Forestalling  in  RAWFS,  whereby  the  decision  maker  makes  a  back-up  plan  in  case  the 
current  best  response  based  on  the  constructed  story  is  wrong.  Overall,  this  method  can  be  thought 
of  as  adding  information  to  the  aggregate  based  on  a  mental  simulation  of  the  scenario  in  order  to 
form  a  judgment  that  can  match  a  previously  experienced  situation.  Therefore  instead  of  searching 
for  additional  information  from  the  environment,  the  current  cues  may  be  re-weighted  to  fit  with  a 
story  representing  the  situation. 

In  addition  to  uncertainty  caused  by  a  lack  of  information,  it  may  also  occur  when  people  are 
presented  with  inconclusive  information.  This  information  may  be  relevant,  but  it  does  not  clearly 
favour  a  particular  belief  (i.e.  friend  or  foe).  Baranksi  (2003)  examined  how  experienced 
Operations  Room  Officers  (OROs)  and  inexperienced  students  aggregated  conclusive  and 
inconclusive  information  in  a  naval  threat  assessment  scenario.  Results  revealed  that  inconsistent 
information  was  not  ignored.  Instead,  it  accentuated  the  hostility  information  for  OROs,  and 
diluted  it  for  students.  Therefore,  experts  erred  on  the  side  of  “foe”  when  inconsistent  information 
was  present  while  inexperienced  participants  erred  on  the  side  of  “friendly”  (Baranksi,  2003). 

These  results  suggest  that  uncertainty  will  be  handled  differently  depending  on  the  level  of 
experience  of  the  decision  maker. 
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5.5  The  presentation  of  information 

The  way  that  information  is  presented  to  the  decision  maker  will  affect  how  the  information  is 
weighed  and  aggregated.  Information  may  have  a  different  value  depending  on  whether  it  is 
presented  simultaneously  or  sequentially  (Banerjeem  1992;  Ottaviani  &  S0rensen,  2001),  as  well  as 
depending  on  the  order  in  which  the  information  is  presented  (Perrin  et  al.,  2001). 

5.5.1  Sequential  Information  Presentation 

Effects  related  to  the  sequential  presentation  of  information  have  already  been  touched  upon  when 
discussing  the  overconfidence  bias  in  Section  5.4.3.  In  that  discussion,  people  performed 
egocentrically  and  chose  to  present  their  personal  opinion  in  a  committee  setting  even  when  they 
should  have  performed  rationally  by  first  updating  their  belief  based  on  information  presented  by 
others.  Here,  we  elaborate  on  the  effects  associated  with  sequential  information  presentation. 

When  people  are  required  to  present  their  opinions  or  judgments  sequentially  in  front  of  each  other, 
they  may  hide  their  information  if  it  does  not  conform  to  the  preceding  opinions.  This  social  factor 
is  called  herding  behaviour  and  it  results  in  the  loss  of  relevant  information  (Banerjeem  1992; 
Ottaviani  &  S0rensen,  2001).  Anti-seniority  rules  have  been  used  to  minimize  this  phenomenon. 

For  example,  hierarchical  and  judicial  systems  often  require  that  senior  experts  speak  later  in  the 
sequence  so  that  more  junior  experts  can  provide  their  opinions  without  being  subject  to  their 
influence  (Ottaviani  &  S0rensen,  2001).  Even  senior  experts  can  fall  trap  to  herding  behaviour 
when  their  opinion  differs  from  the  previous  opinions  (Ottaviani  &  S0rensen,  2001).  Experts 
providing  their  opinion  or  signal  about  the  state  of  the  world,  as  it  is  often  referred  to  in  Economic 
theory,  may  pretend  to  have  observed  a  different  signal  if  they  believe  presenting  the  information 
will  cause  them  to  receive  a  bad  reputation  (Ottaviani  &  S0rensen,  2001).  In  fact,  rational  decision 
making  theory  requires  that  people  conform  to  previous  responses  when  multiple  preceding  opinions 
contrast  with  their  own  opinion.  This  is  due  to  the  assumption  that  contrasting  opinions  must  be 
based  upon  private  information  unavailable  to  the  individual.  Therefore,  they  should  aggregate  this 
information  with  their  own  by  conforming.  This  rational  behaviour  leads  to  the  creation  of 
information  cascades,  where  every  subsequent  speaker,  based  on  observations  of  others,  will  make 
the  same  choice  independent  of  his  private  information  (Noth  &  Weber,  2003).  This  behaviour, 
although  normative  according  to  Bayes’  rule  (therefore  providing  optimal  integration  from  a  rational 
perspective),  is  actually  a  failure  in  aggregating  previously  presented  information  with  one’s  own 
information. 

The  overconfidence  bias  discussed  in  Section  5.4.3,  is  also  a  failure  in  aggregation  because  it 
causes  previous  information  to  be  weighed  too  lightly  (Noth  and  Weber,  2003).  This  bias  leads  to 
a  reversal  of  the  information  cascade  which  can  cause  others  who  are  using  herding  behaviour  to 
erroneously  believe  information  provided  by  the  biased  individual  is  strong  (Noth  &  Weber,  2003). 
Despite  this  possibility,  the  inclusion  of  some  highly  biased  people  may  help  a  discussion  to 
facilitate  acquisition  of  a  maximum  amount  of  information  that  is  otherwise  hidden  within 
individuals  (Ottaviani  &  S0rensen,  2001). 

It  is  difficult  to  extrapolate  this  kind  of  rational  decision  theory  into  a  CID  made  at  a  close  tactical 
level  of  command  due  to  the  unlikelihood  of  a  round-table  discussion  where  each  person  presents 
their  information,  as  often  discussed  in  this  literature.  Nonetheless,  herding  behaviour  may  still 
occur  in  the  absence  of  such  a  “table  discussion”.  For  example,  anecdotal  evidence  shows  that 
subordinate  formations  implicitly  trust  the  plan  provided  by  the  higher  command,  including 
information  about  expected  locations  of  friendly  forces,  control  lines  and  lines  of  furthest  advance. 
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This  leads  to  friendly-fire  incidents  when  other  units  get  lost  and  appear  in  unexpected  locations. 

The  formation  radios  for  additional  information  from  the  operations  officer  who  states  that  no 
friendlies  are  in  the  area,  and  the  formation  opens  fire  on  the  Tost’  unit.  This  same  situation 
occurred  when  a  USAF  A- 10  aircraft  opened  fire  on  a  British  Warrior  armoured  vehicle,  resulting 
in  the  death  of  one  soldier.  The  pilot  radioed  for  confirmation  from  his  controller  and,  despite  his 
own  evidence  to  the  contrary,  opened  fire.  It  is  therefore  necessary  to  investigate  when  and  why 
CID  decision  makers  might  find  themselves  accepting  the  opinion  of  others  at  higher  ranks  despite 
having  conflicting  information  that  could  benefit  the  formation  of  a  judgment. 

5.5.2  Simultaneous  Information  Presentation 

Problems  with  herding  behaviour  can  be  eliminated  by  having  people  provide  their  private  information 
to  the  individual  decision  maker  simultaneously  or  independently  of  others.  For  example,  a  LAV 
commander  might  request  the  opinion  of  the  gunner  regarding  the  identification  of  a  detected  entity 
independently  of  the  opinion  requested  of  an  overhead  pilot  and  the  opinion  of  another  LAV 
commander  closer  to  the  entity.  This  would  reduce  the  likelihood  of  one  of  the  contacts  simply 
agreeing  with  the  others  because  of  herding  behaviour.  It  would  be  necessary  to  determine  whether 
such  information  cascades  could  develop  in  the  high-pressured  military  environment  given  the 
intuitive  nature  of  the  decision.  This  literature  search  did  not  locate  military  research  specific  to  this 
topic. 

Another  benefit  of  viewing  information  simultaneously  or  with  minimal  time  disparity,  is  that  it 
allows  decision  makers  to  make  a  better  judgment  on  the  disagreement  between  information 
sources.  As  already  discussed,  this  will  affect  how  decision  makers  combine  the  information 
because  they  will  be  more  aware  of  dissenting  information  (Budescu  &  Yu,  2006). 

5.5.3  Information  Order  Effects 

Even  when  information  is  presented  simultaneously,  the  human  information  processor  cannot 
consider  all  information  at  exactly  the  same  time  because  of  cognitive  limitations  (e.g.,  attention 
cannot  be  everywhere  at  once,  Wickens  et  al.,  1999).  Therefore,  even  when  information  is  presented 
as  close  as  possible  in  time,  it  will  be  subjected  to  an  ordering.  This  ordering  can  affect  how  the 
information  is  aggregated,  even  when  the  decision  maker  is  an  expert.  Perrin  et  al.  (2001)  performed 
a  study  in  which  sixteen  U.S.  Navy  personnel  determined  the  identity  (civil  or  military)  of 
unidentified  aircraft  using  six  kinds  of  information.  Three  of  the  information  sources  supported  each 
of  the  two  possible  identities.  On  each  trial,  all  of  the  information  was  presented,  but  in  different 
orders.  This  was  done  to  determine  whether  participants  were  using  an  information  order  bias  or 
pattern  matching  when  making  their  judgment.  According  to  Perrin  et  al.  (2001),  pattern  matching 
would  cause  participants  to  update  their  understanding  of  the  situation  as  each  new  piece  of  evidence 
was  provided.  An  information  order  bias,  on  the  other  hand,  would  cause  people  to  dismiss 
information  not  consistent  with  the  judgment  formed.  Perrin  et  al.  (2001)  found  support  for  an 
information  order  bias  because  participants  identified  the  aircraft  as  civil  when  the  first  three  pieces  of 
information  suggested  military  and  the  last  three  suggested  civil.  Therefore,  participants  appeared  to 
be  employing  a  recency  effect  where  the  most  recent  information  was  utilized  for  the  judgment.  This 
effect  is  related  to  memory  processes  which  are  biased  towards  recalling  information  presented  first 
(primacy)  and  last  (recency)  in  sequence  (Hogarth  &  Einhom,  1992  as  cited  in  Wickens  et  al.,  1999). 
Perrin  et  al.  (2001)  suggest  that  the  recency  effect  is  more  likely  in  the  presence  of  a  short,  complex 
sequence  of  evidence  instead  of  a  long  one.  This  is  because  there  will  be  less  pressure  to  alter  the 
judgment  hypothesis  to  fit  with  the  greater  amount  of  information. 
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5.6  Summary 

In  summary,  the  models  and  theories  of  information  aggregation  and  the  various  weighting 
strategies  offer  us  insight  into  the  complexity  involved  when  combining  information.  Although 
much  of  the  literature  is  outside  the  realm  of  military  operations,  we  were  able  to  extrapolate  and 
apply  some  of  the  principles  to  example  CID  scenarios.  Further  research  will  be  necessary  to 
determine  whether  these  extrapolations  are  actually  valid  in  real-life  CID  contexts.  The  following 
and  final  section  of  this  review  will  highlight  the  relationships  between  CID,  intuitive  and  analytic 
decision  theory  and  the  principles  of  information  aggregation  in  order  to  categorize  the  likely  use 
of  these  models. 
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Evaluation  and  Conclusions 


As  determined  by  the  literature,  combat  identification  is  a  complex  decision-making  task  requiring 
successful  combination  of  data-driven  and  cognitively  driven  information  in  a  time -pressured 
environment.  CID  decision  makers  must  reach  a  situation-specific  level  of  certainty  (threshold) 
about  their  identification  before  accepting  it  as  a  final  judgment.  This  certainty  is  affected  by  how 
information  is  weighed  and  combined.  If,  after  acquiring  and  aggregating  some  information,  the 
threshold  of  confidence  is  not  met,  then  more  information  is  sought,  the  situation  awareness  is 
updated,  the  identification  is  revised  (if  necessary),  and  a  new  level  of  confidence  is  reached  (Dean 
et  al.,  2005;  U.S.  Defense  Science  Board,  1996).  If  the  confidence  level  is  adequate,  the  decision¬ 
making  process  can  continue  with  the  selection  and  implementation  of  a  COA  based  on  the 
identification  judgment.  The  necessity  of  achieving  a  faster  OODA  loop  than  the  enemy  (by 
quickly  identifying  the  enemy  and  executing  an  action)  puts  stress  on  the  individual  decision  maker 
to  gather  and  combine  information  rapidly  (Bonifay,  2000).  The  cost  of  making  a  misidentification 
(e.g.,  fratricide,  neutricide  or  suicide)  confounds  this  situation,  creating  a  need  for  a  speed-accuracy 
trade-off  that  will  vary  depending  on  the  intensity  of  the  situation.  For  example,  as  the  distance 
from  a  LAV  and  a  detected  entity  decreases,  the  time  pressure  will  be  enhanced,  therefore 
decreasing  the  threshold  required  to  make  an  identification  (Dean  et  al.,  2005).  Although  a 
decreased  threshold  will  result  in  a  greater  chance  of  fratricide,  the  risk  may  be  necessary  to  avoid 
other  consequences,  such  as  an  unacceptable  risk  to  oneself  or  comrades. 

The  dynamic,  time -pressured  nature  of  CID  and  the  expertise  of  its  decision  makers,  make  it 
difficult  for  this  decision  to  be  compatible  with  the  time-consuming  process  implied  by  analytic 
decision  theory  (ADT).  For  a  soldier  in  the  field,  the  requirement  of  the  ADT  approach  to  use  all 
available  information  and  to  develop  and  consider  all  possible  alternatives  would  not  be  practical. 

In  addition,  the  speed-accuracy  trade  off  required  by  CID  decision  makers  under  many  operational 
circumstances  eliminates  the  need  for  an  optimal  decision.  Instead,  satisficing  is  required  so  that 
an  adequate  identification  can  be  made,  given  the  less  than  adequate  time  available.  If  we  return  to 
the  example  scenario  of  the  search  for  a  Taliban  leader  in  an  Afghan  village,  we  can  recognize  the 
difficulty  associated  with  collecting  all  available  information  about  every  individual  encountered 
before  identifying  them  (i.e.  friend,  foe).  To  do  this,  decision  makers  would  not  only  assess  the 
appearance  and  actions  of  the  individual,  but  would  need  to  determine  the  relationships  between 
the  person  and  others  in  the  village,  the  history  of  the  person,  etc.  This  time  draining  task  would 
surely  allow  for  the  escape  of  the  intended  target.  Instead  of  performing  this  demanding  process, 
CID  decision  makers  need  to  be  able  to  look  at  a  person  and  quickly  categorize  them  based  on  their 
own  understanding  of  the  situation,  the  cues  the  person  presents,  and  in  situations  with  enough 
time,  possibly  use  the  opinions  of  others  and/or  technology. 

Unlike  ADT,  intuitive  decision  theory  offers  a  better  fit  to  CID  decisions  because  of  its  focus  on 
cue-based  decision  making  in  real-life  situations.  Aspects  from  both  BDT  and  NDM  are  helpful  in 
describing  this  decision  process.  For  example,  the  Recognition-Primed  Decision  Model  and  the 
Lens  Model  of  NDM  offer  insight  into  how  experience  fosters  rapid  decision-making  without  using 
demanding  mental  computations.  RPD  suggests  that  people  with  domain-specific  experience 
assess  the  situation  and  then  use  pattern  matching  to  match  cues  from  the  environment  with 
previously  experienced  scenarios  which  are  associated  with  a  particular  action.  This  action  is  then 
mentally  simulated  to  determine  its  effectiveness  in  the  present  situation.  The  Lens  Model,  in 
contrast,  describes  how  experience  and  proper  trust  levels  will  affect  the  calibration  between 
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subjective  weightings  of  cues  and  the  actual  informativeness  of  the  cues.  With  increased 
experience,  people  will  more  easily  allocate  attention  (weight)  to  the  proper  cues,  assuming  they 
are  not  overloaded  by  the  situation.  The  effect  of  factors,  such  as  cognitive  overload,  can  be 
explained  with  the  heuristics  and  cognitive  biases  of  BDT.  By  combining  the  details  of  BDT  with 
the  principles  of  NDM,  C1D  can  be  understood  at  a  greater  level. 

The  models  and  principles  of  information  aggregation  can  also  be  classified  by  their  relationship  to 
analytic  or  intuitive  decision  making  (Table  4).  Methods  of  aggregation  that  are  highly 
computational,  and  aim  for  an  optimal  integration  can  be  considered  as  part  of  the  domain  of 
analytic  decision  theory.  Methods  that  are  used  more  naturally  and  lead  to  a  reasonable 
combination  of  information  fit  within  intuitive  decision  theory. 


Table  4.  Categorization  of  Models  and  Principles  of  Information  Aggregation  Based 

on  Decision  Making  Theory 


Analytic  Decision  Theory 

Intuitive  Decision  Theory 

Information  Integration 

Proper  Linear  Models 

Improper  Linear  Models 

Compensatory  Models 

Non-compensatory  Models 

Cue-abstraction  Model 

Cue-abstraction  Model 

Exemplar  Memory  Model 

Majority  Rule 

Majority  Rule 

Bayes’  Rule 

Naive  Bayes’  Rule 

Averaging  (Larrick  and  Soil,  2006) 

Averaging  (Budescu  &  Yu,  2006) 

Weighted  Average 

Median 

Take-The-Best  Heuristic 

Weighing  information 

Availability  Heuristic 

Representativeness  Heuristic 

Adjustment  and  Anchoring 

Confirmation  Bias 

Consensus  Trimming 

Herding  Behaviour,  Information 
Cascades 

Overconfidence  Bias,  Herding  Behaviour 

Egocentric  Trimming 

“As  If  Heuristic 

Information  Order  Bias 

As  can  be  observed  in  Table  4,  the  majority  of  principles  discussed  in  this  review  fit  under  the 
heading  of  intuitive  decision  theory.  One  reason  for  this  discrepancy  is  that  many  of  the  models 
associated  with  analytic  or  rational  decision  theory  are  too  difficult  for  people  to  perform,  even 
when  given  sufficient  time.  Therefore,  these  methods  are  often  used  as  data  fusion  algorithms, 
which  are  beyond  the  scope  of  this  paper.  For  example,  Dempster-Shafer  Theory  and  Fuzzy  Logic 
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are  both  methods  of  information  integration  that  should  fall  within  the  analytic  decision  category 
because  of  their  formation  of  an  optimal  aggregate.  However,  literature  could  not  be  found 
suggesting  the  natural,  unassisted  use  of  these  methods  by  humans. 

Several  of  the  methods  of  information  aggregation  are  included  under  both  analytic  and  intuitive 
decision  theory.  This  duplication  is  caused  by  conflicting  literature  regarding  the  natural  use  of 
these  methods  in  time-restricted  situations.  For  example,  the  majority  rule  is  designed  to  create  the 
optimal  integration  of  information  dispersed  across  individuals,  therefore  making  it  an  analytic 
approach  (Piketty,  1999).  At  the  same  time,  it  is  common  for  people  to  seek  consensus  information 
in  dynamic  situations  by  ignoring  conflicting  information,  and  in  turn  accepting  the  “majority” 
response.  In  other  situations,  however,  dissenting  information  may  be  used  because  of  the 
expertise  or  previous  accuracy  of  that  source,  therefore  violating  the  majority  rule  (Harries  et  al., 
2004).  Cue-abstraction  is  also  listed  under  both  categories.  This  is  because  it  involves  the 
governance  of  cue  weightings  based  on  learned  rules,  which  is  not  always  possible  in  natural 
settings.  There  may  be  some  situations  where  there  are  well-established  rules  regarding  the 
relationship  between  cues  and  criterion.  However,  in  many  cases,  especially  battlefield  scenarios, 
the  cue-criterion  relationships  change  too  frequently  to  form  a  set  of  rules. 

By  assuming  that  CID  decision  makers  will  be  subject  to  the  same  restrictions  and  biases 
associated  with  intuitive  decision  theory,  we  can  summarize  the  applicable  methods  of  weighing 
and  combining  information  as  follows: 

•  Pattern  matching  between  observed  cues  and  mental  schemas  or  prototypes  may  assist 
in  understanding  the  situation. 

•  Pattern  matching  between  observed  cues  and  memorized  exemplars  might  assist  in 
forming  a  combat  identification. 

•  The  identification  threshold  (confidence  level)  will  be  lowered  if  it  is  easy  to  recall  an 
instance  of  enemy  contact.  For  example,  if  a  soldier  has  a  vivid  memory  of  an 
encounter  with  an  enemy,  he  may  be  more  likely  to  accept  an  enemy  identification 
with  a  lower  level  of  certainty  than  to  risk  this  kind  of  misidentification.  This  is  an 
example  of  the  availability  heuristic. 

•  The  identification  threshold  will  be  raised  if  it  is  difficult  to  recall  an  instance  of  enemy 
contact.  For  example,  if  a  soldier  has  no  memory  of  enemy  encounters,  he  may  require 
more  certainty  that  information  is  guiding  him  towards  an  enemy  identification  before 
stopping  the  cue  acquisition  phase.  This  is  an  example  of  the  availability  heuristic. 

•  Weights  will  be  assigned  to  cues  non-optimally  based  on  heuristics  and  cognitive 
biases  due  to  the  uncertainty  and  time  pressure  in  the  battlefield. 

•  Soldiers,  who  have  experienced  accurate  and  reliable  information  from  information 
sources  in  the  past,  such  as  from  IFF  sensors,  will  put  more  weight  on  this  information 
when  forming  their  judgment. 

•  Trust  levels  gained  through  experience  will  be  ignored  when  the  situation  is  too 
cognitively  demanding,  causing  each  cue  to  be  weighed  equally. 

•  Soldiers  may  ignore  the  opinions  of  others  in  favour  of  their  own  beliefs  about  the 
identity  of  an  entity  because  of  an  overconfidence  bias.  They  may  also  seek  to  prove 
their  judgment  by  ignoring  conflicting  information  because  of  a  confirmatory  bias. 
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•  Only  a  single  cue  may  be  used  to  form  a  judgment  if  the  decision  maker  views  it  as 
adequate  for  distinguishing  between  two  options  (e.g.,  friend  vs.  unknown;  enemy  vs. 
not  an  enemy).  This  is  an  example  of  using  the  Take-the-Best  Heuristic. 

•  If  cues  suggest  different  identifications,  the  most  recently  presented  information  may 
be  used  to  form  the  judgment  because  of  an  information  order  bias. 

•  If  an  information  source  is  known  to  use  multiple  cues  to  form  a  judgment,  this  source 
will  receive  a  greater  weighting  when  forming  ones’  own  judgment.  This  is  an 
example  of  the  weighted  average  strategy >. 

•  If  an  information  source  provides  a  numeric  estimate,  such  as  blue  force  position 
coordinates,  the  soldier  will  anchor  their  estimate  of  this  position  based  on  this 
information,  even  when  they  know  it  is  erroneous.  This  is  an  example  of  the 
Adjustment  and  Anchoring  Heuristic. 

•  In  a  situation  where  time  permits  communication  of  opinions  about  the  identity  of  an 
entity,  the  majority  “vote”  may  be  accepted,  unless  dissenting  judgments  are  made  by  a 
consistently  accurate,  but  outlying  source. 

•  In  a  situation  where  time  permits  communication  of  opinions  about  the  identity  of  an 
entity,  the  soldier  may  accept  the  opinion  of  superiors  despite  possessing  conflicting 
information.  This  is  an  example  of  herding  behaviour. 

Studies,  such  as  battle  simulations  will  be  necessary  to  determine  the  validity  of  these  methods  of 
information  aggregation  as  they  apply  to  CID.  It  should  also  be  noted  that  in  order  to  make  a 
successful  combat  identification,  decision  makers  must  not  only  accurately  weigh  and  combine 
information,  but  they  need  to  first  utilize  their  experience  to  gather  the  appropriate  information,  and 
following  the  aggregation  they  must  have  an  efficient  strategy  for  identifying  and  making  use  of  a 
course  of  action.  Information  aggregation  is  only  one  part  of  this  complex  process. 
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In  conclusion,  this  report  has  reviewed  the  literature  for  models  of  human  information  aggregation 
in  order  to  relate  it  to  combat  identification  decisions  made  by  individuals  in  a  land  force  context. 
Although  the  literature  related  to  models  of  information  aggregation  was  limited  by  its  primary 
application  to  economics,  political  science  and  organization  behaviour,  we  were  able  to  extrapolate 
a  variety  of  principles  to  facilitate  understanding  of  the  processes  involved  in  the  weighing  and 
combining  of  information  during  the  formation  of  a  belief  or  judgment.  The  review  found  that  the 
characteristics  of  combat  identification  fit  it  to  the  intuitive  decision  making  framework.  By 
determining  the  methods  of  aggregation  associated  with  this  framework  we  were  able  to  form  a  list 
of  principles,  heuristics  and  biases  that  may  affect  CID  decision  makers. 
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ID 
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IV 
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Operations  Room  Operator 
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Princess  Patricia’s  Canadian  Light  Infantry 

ROE 

Rules  of  Engagement 

RPD 

Recognition-Primed  Decision 

SA 

Situational  Awareness 
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Trust  Bias 
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Trust  Calibration 

TI 

Target  Identification 
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